Fix tests to work with AGASC 1.8

[taldcroft]

Description

Fixes tests so that they pass with AGASC 1.8 available as the default flight AGASC.

Fixes #159

Interface impacts

None

Testing

Made a new local directory agasc_test_1p8 with a symlink of every file in $SKA/data/agasc. In that directory I made two more symlinks agasc1p8.h5 -> agasc1p8rc11.h5 and proseco_agasc_1p8.h5 -> proseco_agasc_1p8rc11.h5.

Without this patch there are 3 tests that fail for the AGASC 1.8 case.

With this patch:

(ska3) ➜  agasc git:(fix-tests-for-agasc1p8) env AGASC_DIR=$PWD/agasc_test_1p8 AGASC_HDF5_FILE=junk pytest
====================================================== test session starts ======================================================
platform darwin -- Python 3.11.8, pytest-7.4.4, pluggy-1.4.0
rootdir: /Users/aldcroft/git
configfile: pytest.ini
plugins: timeout-2.2.0, anyio-4.3.0
collected 75 items                                                                                                              

agasc/tests/test_agasc_1.py .....                                                                                         [  6%]
agasc/tests/test_agasc_2.py ..........sssss..........ss..................                                                 [ 66%]
agasc/tests/test_agasc_healpix.py ...........                                                                             [ 81%]
agasc/tests/test_obs_status.py ..............                                                                             [100%]

================================================= 68 passed, 7 skipped in 7.06s =================================================
(ska3) ➜  agasc git:(fix-tests-for-agasc1p8) env AGASC_DIR=$SKA/data/agasc AGASC_HDF5_FILE=junk pytest 
====================================================== test session starts ======================================================
platform darwin -- Python 3.11.8, pytest-7.4.4, pluggy-1.4.0
rootdir: /Users/aldcroft/git
configfile: pytest.ini
plugins: timeout-2.2.0, anyio-4.3.0
collected 75 items                                                                                                              

agasc/tests/test_agasc_1.py .....                                                                                         [  6%]
agasc/tests/test_agasc_2.py ..........sssss..........ss..................                                                 [ 66%]
agasc/tests/test_agasc_healpix.py ...........                                                                             [ 81%]
agasc/tests/test_obs_status.py ..............                                                                             [100%]

================================================= 68 passed, 7 skipped in 7.93s =================================================
(ska3) ➜  agasc git:(fix-tests-for-agasc1p8) git rev-parse HEAD  
99493884f148f779fe196b54b92280bc9b66699d

Unit tests

• Mac

Independent check of unit tests by Jean

• Linux - I also copied the 1p7 files into /proj/sot/ska/data/agasc/rc so it would run all the healpix tests.

ska3-jeanconn-fido> echo $AGASC_DIR
/proj/sot/ska/data/agasc/rc
ska3-jeanconn-fido> git rev-parse HEAD
99493884f148f779fe196b54b92280bc9b66699d
ska3-jeanconn-fido> pytest
============================================================== test session starts ===============================================================
platform linux -- Python 3.11.8, pytest-7.4.4, pluggy-1.4.0
rootdir: /proj/sot/ska/jeanproj/git
configfile: pytest.ini
plugins: anyio-4.3.0, timeout-2.2.0
collected 75 items                                                                                                                               

agasc/tests/test_agasc_1.py .....                                                                                                          [  6%]
agasc/tests/test_agasc_2.py .............................................                                                                  [ 66%]
agasc/tests/test_agasc_healpix.py ...........                                                                                              [ 81%]
agasc/tests/test_obs_status.py ..............                                                                                              [100%]

========================================================= 75 passed in 97.38s (0:01:37)

Functional tests

No functional testing.

[jeanconn]

This PR fixes a bunch of the tests, but some still need update.

export AGASC_DIR=/proj/sot/ska/data/agasc/rc
ska3-jeanconn-fido> pytest
================================================ test session starts =================================================
platform linux -- Python 3.11.8, pytest-7.4.4, pluggy-1.4.0
rootdir: /proj/sot/ska/jeanproj/git
configfile: pytest.ini
plugins: anyio-4.3.0, timeout-2.2.0
collected 75 items                                                                                                                               

agasc/tests/test_agasc_1.py .....                                                                                                          [  6%]
agasc/tests/test_agasc_2.py ............................FF..F............                                                                  [ 66%]
agasc/tests/test_agasc_healpix.py .ssssssssss                                                                                              [ 81%]
agasc/tests/test_obs_status.py ..............                                                                                              [100%]

==================================================================== FAILURES ====================================================================
_________________________________________________________ test_supplement_get_agasc_cone _________________________________________________________

    @pytest.mark.skipif(NO_MAGS_IN_SUPPLEMENT, reason="no mags in supplement")
    def test_supplement_get_agasc_cone():
        ra, dec = 282.53, -0.38  # Obsid 22429 with a couple of color1=1.5 stars
        stars1 = agasc.get_agasc_cone(
            ra, dec, date="2021:001", agasc_file=MINIAGASC, use_supplement=False
        )
        stars2 = agasc.get_agasc_cone(
            ra, dec, date="2021:001", agasc_file=MINIAGASC, use_supplement=True
        )
        ok = stars2["MAG_CATID"] == agasc.MAG_CATID_SUPPLEMENT
    
        change_names = ["MAG_CATID", "COLOR1", "MAG_ACA", "MAG_ACA_ERR"]
        for name in set(stars1.colnames) - set(change_names):
            assert np.all(stars1[name] == stars2[name])
    
        assert not np.any(stars1["MAG_CATID"] == agasc.MAG_CATID_SUPPLEMENT)
    
        # At least 35 stars in this field observed
        assert np.count_nonzero(ok) >= 35
    
        # At least 7 color=1.5 stars converted to 1.49 (note: total of 23 color=1.5
        # stars in this field)
        assert np.count_nonzero(stars1["COLOR1"][ok] == 1.49) == 0
>       assert np.count_nonzero(stars1["COLOR1"][ok] == 1.50) >= 7
E       AssertionError: assert 1 >= 7
E        +  where 1 = <function count_nonzero at 0x7f80fef806b0>(<Column name='COLOR1' dtype='float32' length=37>\n0.1796875\n2.9726562\n 2.578125\n  0.84375\n0.7890625\n0.1640625\n 2.078125...2.9414062\n0.4765625\n1.2226562\n 0.140625\n4.1289062\n0.9140625\n 1.265625\n     1.25\n3.1914062\n  0.59375\n 0.078125\n 1.421875 == 1.5)
E        +    where <function count_nonzero at 0x7f80fef806b0> = np.count_nonzero

agasc/tests/test_agasc_2.py:442: AssertionError
____________________________________________________________ test_supplement_get_star ____________________________________________________________

    @pytest.mark.skipif(NO_MAGS_IN_SUPPLEMENT, reason="no mags in supplement")
    def test_supplement_get_star():
        agasc_id = 58720672
        # Also checks that the default is False given the os.environ override for
        # this test file.
        star1 = agasc.get_star(agasc_id, agasc_file=MINIAGASC)
        star2 = agasc.get_star(agasc_id, agasc_file=MINIAGASC, use_supplement=True)
        assert star1["MAG_CATID"] != agasc.MAG_CATID_SUPPLEMENT
        assert star2["MAG_CATID"] == agasc.MAG_CATID_SUPPLEMENT
    
        assert star1["AGASC_ID"] == star2["AGASC_ID"]
    
>       assert np.isclose(star1["COLOR1"], 1.50)
E       assert False
E        +  where False = <function isclose at 0x7f80fef81a30>(1.9921875, 1.5)
E        +    where <function isclose at 0x7f80fef81a30> = np.isclose

agasc/tests/test_agasc_2.py:471: AssertionError
___________________________________________________________ test_supplement_get_stars ____________________________________________________________

    @pytest.mark.skipif(NO_MAGS_IN_SUPPLEMENT, reason="no mags in supplement")
    def test_supplement_get_stars():
        agasc_ids = [58720672, 670303120]
        star1 = agasc.get_stars(agasc_ids, agasc_file=MINIAGASC)
        star2 = agasc.get_stars(agasc_ids, agasc_file=MINIAGASC, use_supplement=True)
        assert np.all(star1["MAG_CATID"] != agasc.MAG_CATID_SUPPLEMENT)
        assert np.all(star2["MAG_CATID"] == agasc.MAG_CATID_SUPPLEMENT)
    
        assert np.all(star1["AGASC_ID"] == star2["AGASC_ID"])
    
>       assert np.allclose(star1["COLOR1"], [1.5, 0.24395067])
E       AssertionError: assert False
E        +  where False = <function allclose at 0x7f80fef818b0>(<Column name='COLOR1' dtype='float32' length=2>\n1.9921875\n0.3359375, [1.5, 0.24395067])
E        +    where <function allclose at 0x7f80fef818b0> = np.allclose

agasc/tests/test_agasc_2.py:514: AssertionError
============================================================ short test summary info =============================================================
FAILED agasc/tests/test_agasc_2.py::test_supplement_get_agasc_cone - AssertionError: assert 1 >= 7
FAILED agasc/tests/test_agasc_2.py::test_supplement_get_star - assert False
FAILED agasc/tests/test_agasc_2.py::test_supplement_get_stars - AssertionError: assert False
============================================== 3 failed, 62 passed, 10 skipped in 96.90s (0:01:36)```

[taldcroft]

Good catch @jeanconn. I made an additional fix and hopefully it all passes for you now.