Question about local test error message

[martinholmer]

I'm getting an error I don't understand when I do "make pytest-cps" on my computer using the code on the master branch. What am I doing wrong?

(taxcalc-dev) Tax-Calculator% git checkout master
Switched to branch 'master'
Your branch is up to date with 'origin/master'.
(taxcalc-dev) Tax-Calculator% ./gitsync 
remote: Enumerating objects: 286, done.
remote: Counting objects: 100% (273/273), done.
remote: Compressing objects: 100% (58/58), done.
remote: Total 170 (delta 136), reused 142 (delta 112), pack-reused 0
Receiving objects: 100% (170/170), 90.68 KiB | 3.36 MiB/s, done.
Resolving deltas: 100% (136/136), completed with 80 local objects.
From https://github.com/PSLmodels/Tax-Calculator
   4cb3bad3..3aecc3cb  gh-pages   -> upstream/gh-pages
   61251f60..6528ca7b  master     -> upstream/master
Updating 61251f60..6528ca7b
Fast-forward
 docs/about/license.md => LICENSE.md        |    0
 docs/_toc.yml                              |    6 +-
 docs/about/LICENSE.md                      |   22 +
 docs/guide/policy_params.md                | 9136 ++++++++++++++--------------
 taxcalc/calcfunctions.py                   |   20 +-
 taxcalc/policy_current_law.json            |   47 +
 taxcalc/tests/test_stats_benchmark.csv     |  335 -
 taxcalc/utils.py                           |    2 +-
 taxcalc/validation/taxsim35/input_setup.py |    2 +-
 9 files changed, 4662 insertions(+), 4908 deletions(-)
 rename docs/about/license.md => LICENSE.md (100%)
 create mode 100644 docs/about/LICENSE.md
 delete mode 100644 taxcalc/tests/test_stats_benchmark.csv
Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
To https://github.com/martinholmer/Tax-Calculator.git
   61251f60..6528ca7b  master -> master

(taxcalc-dev) Tax-Calculator% make pytest-cps 
//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pep8.py:110: FutureWarning: Possible nested set at position 1
  EXTRANEOUS_WHITESPACE_REGEX = re.compile(r'[[({] | []}),;:]')
============================= test session starts ==============================
platform darwin -- Python 3.11.7, pytest-7.4.4, pluggy-1.0.0
rootdir: /Users/mrh/work/Tax-Calculator
configfile: pytest.ini
plugins: harvest-1.10.4, xdist-3.5.0, pep8-1.0.6
initialized: 4/4 workers//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pep8.py:110: FutureWarning: Possible nested set at position 1
  EXTRANEOUS_WHITESPACE_REGEX = re.compile(r'[[({] | []}),;:]')
//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pep8.py:110: FutureWarning: Possible nested set at position 1
  EXTRANEOUS_WHITESPACE_REGEX = re.compile(r'[[({] | []}),;:]')
//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pep8.py:110: FutureWarning: Possible nested set at position 1
  EXTRANEOUS_WHITESPACE_REGEX = re.compile(r'[[({] | []}),;:]')
//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pep8.py:110: FutureWarning: Possible nested set at position 1
  EXTRANEOUS_WHITESPACE_REGEX = re.compile(r'[[({] | []}),;:]')
4 workers [278 items]   
........................................................................ [ 25%]
........................................................................ [ 51%]
........................................................................ [ 77%]
..............................................................           [100%]Traceback (most recent call last):
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/bin/pytest", line 10, in <module>
    sys.exit(console_main())
             ^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/_pytest/config/__init__.py", line 192, in console_main
    code = main()
           ^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/_pytest/config/__init__.py", line 169, in main
    ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pluggy/_callers.py", line 60, in _multicall
    return outcome.get_result()
           ^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/_pytest/main.py", line 318, in pytest_cmdline_main
    return wrap_session(config, _main)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/_pytest/main.py", line 306, in wrap_session
    config.hook.pytest_sessionfinish(
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pluggy/_callers.py", line 55, in _multicall
    gen.send(outcome)
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/_pytest/terminal.py", line 857, in pytest_sessionfinish
    outcome.get_result()
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mrh/work/Tax-Calculator/taxcalc/tests/conftest.py", line 146, in pytest_sessionfinish
    old_stats_df = pandas.read_csv(os.path.join(tests_path, 'test_stats_benchmark.csv'))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pandas/io/parsers/readers.py", line 948, in read_csv
    return _read(filepath_or_buffer, kwds)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pandas/io/parsers/readers.py", line 611, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pandas/io/parsers/readers.py", line 1448, in __init__
    self._engine = self._make_engine(f, self.engine)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pandas/io/parsers/readers.py", line 1705, in _make_engine
    self.handles = get_handle(
                   ^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pandas/io/common.py", line 863, in get_handle
    handle = open(
             ^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 
  '/Users/mrh/work/Tax-Calculator/taxcalc/tests/test_stats_benchmark.csv'
make: *** [pytest-cps] Error 1
(taxcalc-dev) Tax-Calculator% 

[jdebacker]

@martinholmer Thanks for flagging this. It's coming from a file missing that should be there. This came from me reviewing PR #2709 too quickly and not catching the file deletion included in that PR.

[jdebacker]

@martinholmer I just merged PR #2712, so if you sync to master I do not think you will run into the issues noted here.

[martinholmer]

@jdebacker, Thanks for the quick response to issue #2711.

I don't get any errors now, but I do get a ton of warnings.
See below for the different types of warnings.

First question:
Why are we timing the tests and where do the benchmark test execution times come from?

Second question:
Why am I getting this pytest error? Does it have to do with the timing of the tests?

Traceback (most recent call last):
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/bin/pytest", line 10, in <module>
    sys.exit(console_main())
             ^^^^^^^^^^^^^^

Here is the complete (and highly confusing) results of "make pytest-cps":

(taxcalc-dev) Tax-Calculator% ./gitsync 
remote: Enumerating objects: 22, done.
remote: Counting objects: 100% (22/22), done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 13 (delta 9), reused 10 (delta 7), pack-reused 0
Unpacking objects: 100% (13/13), 11.49 KiB | 1.15 MiB/s, done.
From https://github.com/PSLmodels/Tax-Calculator
   6528ca7b..552b2f88  master     -> upstream/master
Updating 6528ca7b..552b2f88
Fast-forward
 taxcalc/tests/test_stats_benchmark.csv | 335 +++++++++++++++++++++++++++++++++
 taxcalc/utils.py                       |   4 +-
 2 files changed, 337 insertions(+), 2 deletions(-)
 create mode 100644 taxcalc/tests/test_stats_benchmark.csv
Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
To https://github.com/martinholmer/Tax-Calculator.git
   6528ca7b..552b2f88  master -> master

(taxcalc-dev) Tax-Calculator% make pytest-cps
//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pep8.py:110: FutureWarning: Possible nested set at position 1
  EXTRANEOUS_WHITESPACE_REGEX = re.compile(r'[[({] | []}),;:]')
============================= test session starts ==============================
platform darwin -- Python 3.11.7, pytest-7.4.4, pluggy-1.0.0
rootdir: /Users/mrh/work/Tax-Calculator
configfile: pytest.ini
plugins: harvest-1.10.4, xdist-3.5.0, pep8-1.0.6
initialized: 4/4 workers//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pep8.py:110: FutureWarning: Possible nested set at position 1
  EXTRANEOUS_WHITESPACE_REGEX = re.compile(r'[[({] | []}),;:]')
//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pep8.py:110: FutureWarning: Possible nested set at position 1
  EXTRANEOUS_WHITESPACE_REGEX = re.compile(r'[[({] | []}),;:]')
//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pep8.py:110: FutureWarning: Possible nested set at position 1
  EXTRANEOUS_WHITESPACE_REGEX = re.compile(r'[[({] | []}),;:]')
//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pep8.py:110: FutureWarning: Possible nested set at position 1
  EXTRANEOUS_WHITESPACE_REGEX = re.compile(r'[[({] | []}),;:]')
4 workers [278 items]   
........................................................................ [ 25%]
........................................................................ [ 51%]
........................................................................ [ 77%]
..............................................................           [100%]taxcalc/tests/test_calculator.py::test_calculator_mtr is slower than the current benchmark by 7568.432 ms
taxcalc/tests/test_calculator.py::test_ID_HC_vs_BS is slower than the current benchmark by 1146.178 ms
taxcalc/tests/test_calculator.py::test_ID_StateLocal_HC_vs_CRT is slower than the current benchmark by 1678.292 ms
taxcalc/tests/test_calculator.py::test_ID_RealEstate_HC_vs_CRT is slower than the current benchmark by 1882.211 ms
taxcalc/tests/test_policy.py::TestAdjust::test_apply_cpi_offset is slower than the current benchmark by 1326.902 ms
taxcalc/tests/test_taxcalcio.py::test_output_options is slower than the current benchmark by 3920.533 ms
taxcalc/tests/test_taxcalcio.py::test_no_tables_or_graphs is slower than the current benchmark by 1019.528 ms
taxcalc/tests/test_calcfunctions.py::test_EITCamount[test_tuple0-6660] is slower than the current benchmark by 2047.391 ms
taxcalc/tests/test_calculator.py::test_qbid_limit_switch is slower than the current benchmark by 20668.816 ms
taxcalc/tests/test_calculator.py::test_calc_all_benefits_amounts is slower than the current benchmark by 1168.572 ms
taxcalc/tests/test_consumption.py::test_consumption_response is slower than the current benchmark by 1126.08 ms
taxcalc/tests/test_cpscsv.py::test_agg is slower than the current benchmark by 13421.672 ms
taxcalc/tests/test_policy.py::test_index_offset_reform is slower than the current benchmark by 1045.506 ms
taxcalc/tests/test_reforms.py::test_round_trip_reforms[2019] is slower than the current benchmark by 2254.137 ms
taxcalc/tests/test_reforms.py::test_round_trip_reforms[2020] is slower than the current benchmark by 1798.684 ms
taxcalc/tests/test_reforms.py::test_round_trip_reforms[2021] is slower than the current benchmark by 2165.293 ms
taxcalc/tests/test_taxcalcio.py::test_write_doc_file is slower than the current benchmark by 1906.345 ms
taxcalc/tests/test_calculator.py::test_reform_documentation is slower than the current benchmark by 1387.016 ms
taxcalc/tests/test_calculator.py::test_distribution_tables is slower than the current benchmark by 21282.246 ms
taxcalc/tests/test_calculator.py::test_ce_aftertax_income is slower than the current benchmark by 1155.073 ms
taxcalc/tests/test_policy.py::TestAdjust::test_multiple_cpi_swaps2 is slower than the current benchmark by 1060.249 ms
taxcalc/tests/test_policy.py::TestAdjust::test_adj_CPI_offset_and_index_status is slower than the current benchmark by 1213.617 ms
taxcalc/tests/test_policy.py::TestAdjust::test_cpi_offset_does_not_affect_wage_indexed_params is slower than the current benchmark by 1273.254 ms
taxcalc/tests/test_reforms.py::test_round_trip_reforms[2022] is slower than the current benchmark by 2263.939 ms
taxcalc/tests/test_reforms.py::test_round_trip_reforms[2023] is slower than the current benchmark by 1899.54 ms
taxcalc/tests/test_reforms.py::test_reform_json_and_output is slower than the current benchmark by 8708.887 ms
taxcalc/tests/test_benefits.py::test_benefits is slower than the current benchmark by 1901.282 ms
taxcalc/tests/test_parameters.py::test_json_file_contents[policy_current_law.json] is slower than the current benchmark by 15853.787 ms
taxcalc/tests/test_taxcalcio.py::test_creation_with_aging is slower than the current benchmark by 2442.26 ms
taxcalc/tests/test_taxcalcio.py::test_ctor_init_with_cps_files is slower than the current benchmark by 6147.77 ms
taxcalc/tests/test_taxcalcio.py::test_graphs is slower than the current benchmark by 20176.115 ms



=============================== warnings summary ===============================
Traceback (most recent call last):
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/bin/pytest", line 10, in <module>
    sys.exit(console_main())
             ^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/_pytest/config/__init__.py", line 192, in console_main
    code = main()
           ^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/_pytest/config/__init__.py", line 169, in main
    ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pluggy/_callers.py", line 60, in _multicall
    return outcome.get_result()
           ^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/_pytest/main.py", line 318, in pytest_cmdline_main
    return wrap_session(config, _main)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/_pytest/main.py", line 306, in wrap_session
    config.hook.pytest_sessionfinish(
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pluggy/_callers.py", line 55, in _multicall
    gen.send(outcome)
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/_pytest/terminal.py", line 867, in pytest_sessionfinish
    self.config.hook.pytest_terminal_summary(
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pluggy/_callers.py", line 60, in _multicall
    return outcome.get_result()
           ^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pluggy/_callers.py", line 34, in _multicall
    next(gen)  # first yield
    ^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/_pytest/terminal.py", line 883, in pytest_terminal_summary
    self.summary_warnings()
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/_pytest/terminal.py", line 997, in summary_warnings
    maybe_location = collapsed_location_report(message_reports)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/_pytest/terminal.py", line 979, in collapsed_location_report
    location = w.get_location(self.config)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/_pytest/terminal.py", line 337, in get_location
    relpath = bestrelpath(config.invocation_params.dir, absolutepath(filename))
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/_pytest/pathlib.py", line 770, in bestrelpath
    reldest = dest.relative_to(base)
              ^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/pathlib.py", line 730, in relative_to
    raise ValueError("{!r} is not in the subpath of {!r}"
ValueError: '//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/fsspec/registry.py' is not in the subpath of '/Users/mrh' OR one path is relative and the other is absolute.
make: *** [pytest-cps] Error 1
(taxcalc-dev) Tax-Calculator% >....                                             
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/pluggy/_callers.py", line 34, in _multicall
    next(gen)  # first yield
    ^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/_pytest/terminal.py", line 883, in pytest_terminal_summary
    self.summary_warnings()
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/_pytest/terminal.py", line 997, in summary_warnings
    maybe_location = collapsed_location_report(message_reports)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/_pytest/terminal.py", line 979, in collapsed_location_report
    location = w.get_location(self.config)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/_pytest/terminal.py", line 337, in get_location
    relpath = bestrelpath(config.invocation_params.dir, absolutepath(filename))
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/_pytest/pathlib.py", line 770, in bestrelpath
    reldest = dest.relative_to(base)
              ^^^^^^^^^^^^^^^^^^^^^^
  File "//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/pathlib.py", line 730, in relative_to
    raise ValueError("{!r} is not in the subpath of {!r}"
ValueError: '//Users/mrh/anaconda3/envs/taxcalc-dev/lib/python3.11/site-packages/fsspec/registry.py' is not in the subpath of '/Users/mrh' OR one path is relative and the other is absolute.
make: *** [pytest-cps] Error 1

[jdebacker]

@martinholmer asks:

First question:
Why are we timing the tests and where do the benchmark test execution times come from?

The performance benchmarking was introduced in PR #2570 to provide automated profiling of changes in performance for new PRs. At the time, work was being done to consider vectorized code with Dask arrays rather than Numba-jitted functions, so this was especially relevant. The test execution times are reported from pytest and gathered and formatted in Line 138 of conftest.py.

Second question:
Why am I getting this pytest error? Does it have to do with the timing of the tests?

I do not know. It looks like all the tests in pytest passed for you. I don't know what to mark of the warnings summary. I actually have a much longer list of warnings, with notes about future deprecations in the Pandas API, etc.

[martinholmer]

@jdebacker said in issue #2711:

The performance benchmarking was introduced in PR #2570 to provide automated profiling of changes in performance for new PRs. At the time, work was being done to consider vectorized code with Dask arrays rather than Numba-jitted functions, so this was especially relevant.

OK. Is a change from numba to dask still under consideration? If not, can we somehow skip the pytest_sessionfinish because it keeps leaving a file on my disk that is not part of the repository. This is just harassment of developers whose only sin is to have a slower computer than the one used to establish the "benchmark" timings. After running "make pytest-cps", running "git status" produces this:

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	taxcalc/tests/test_stats_current.csv

What am I supposed to do with this untracked file?