Exception: Cannot locate xUnit report / IOError: [Errno 21] Is a directory

[peterjc]

Like #724, but the circumstantial evidence points are a different root cause:

See the following TravisCI tests from the same tool commit peterjc/galaxy_blast@3095f19 run as TravisCI cron jobs:

• https://travis-ci.org/peterjc/galaxy_blast/builds/552875531 (worked, approx 1 July 2019)
• https://travis-ci.org/peterjc/galaxy_blast/builds/555896113 (failed with Galaxy dev branch, approx 8 July 2019)

Traceback (most recent call last):
  File "./scripts/functional_tests.py", line 126, in <module>
    driver_util.drive_test(find_test_driver())
  File "/home/travis/build/peterjc/galaxy_blast/galaxy-dev/test/base/driver_util.py", line 1024, in drive_test
    sys.exit(test_driver.run())
  File "/home/travis/build/peterjc/galaxy_blast/galaxy-dev/test/base/driver_util.py", line 835, in run
    self.setup()
  File "/home/travis/build/peterjc/galaxy_blast/galaxy-dev/test/base/driver_util.py", line 898, in setup
    self._register_and_run_servers(config_object)
  File "/home/travis/build/peterjc/galaxy_blast/galaxy-dev/test/base/driver_util.py", line 953, in _register_and_run_servers
    self.app = build_galaxy_app(galaxy_config)
  File "/home/travis/build/peterjc/galaxy_blast/galaxy-dev/test/base/driver_util.py", line 563, in build_galaxy_app
    app = GalaxyUniverseApplication(**simple_kwargs)
  File "/home/travis/build/peterjc/galaxy_blast/galaxy-dev/lib/galaxy/app.py", line 136, in __init__
    self.data_managers = DataManagers(self)
  File "/home/travis/build/peterjc/galaxy_blast/galaxy-dev/lib/galaxy/tools/data_manager/manager.py", line 38, in __init__
    self.load_from_xml(filename)
  File "/home/travis/build/peterjc/galaxy_blast/galaxy-dev/lib/galaxy/tools/data_manager/manager.py", line 48, in load_from_xml
    tree = util.parse_xml(xml_filename)
  File "/home/travis/build/peterjc/galaxy_blast/galaxy-dev/lib/galaxy/util/__init__.py", line 236, in parse_xml
    root = tree.parse(fname, parser=ElementTree.XMLParser(target=DoctypeSafeCallbackTarget()))
  File "/opt/python/2.7.14/lib/python2.7/xml/etree/ElementTree.py", line 647, in parse
    source = open(source, "rb")
IOError: [Errno 21] Is a directory: './'
Testing complete. HTML report is in "/home/travis/build/peterjc/galaxy_blast/tool_test_output.html".
Cannot locate xUnit report [/tmp/tmpLlbDlG/xunit.xml] for tests - required to build planemo report and summarize tests.
Traceback (most recent call last):
  File "/home/travis/virtualenv/python2.7.14/bin/planemo", line 10, in <module>
    sys.exit(planemo())
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/planemo/cli.py", line 189, in handle_blended_options
    return f(*args, **kwds)
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/planemo/commands/cmd_test.py", line 93, in cli
    return_value = run_in_config(ctx, config, **kwds)
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/planemo/galaxy/test/actions.py", line 102, in run_in_config
    _check_test_outputs(xunit_report_file_tracker, structured_report_file_tracker)
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/planemo/galaxy/test/actions.py", line 283, in _check_test_outputs
    raise Exception(message)
Exception: Cannot locate xUnit report [/tmp/tmpLlbDlG/xunit.xml] for tests - required to build planemo report and summarize tests.
The command "planemo test --no_conda_auto_install --galaxy_root ${TRAVIS_BUILD_DIR}/galaxy-${GALAXY_BRANCH} --skip_venv tools/" exited with 1.

[jdavcs]

Same error+exception happening when running locally (planemo test on galaxy dev branch). No issue when running on release_19.05. I tried removing *pyc files as per #724 (comment) - which did not fix it. I'm looking into it...

[jdavcs]

Here's why the IOError happens:
This line: https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/tools/data_manager/manager.py#L48
expects xml_filename to be a path to a file. If the file does not exist, an exception is not raised (it's expected behavior). However, when galaxy is managed by planemo, xml_filename is a directory name, so e.errno is 21 (vs 2), and an exception is raised. This works under release_19.05 because it has less stricter exception handling than dev: https://github.com/galaxyproject/galaxy/blob/release_19.05/lib/galaxy/tools/data_manager/manager.py#L44

I don't have a solution yet, but I should have one soon.

ping @natefoo : I believe the following commit is 3 years old, but was only recently merged, which is why the issue never surfaced? galaxyproject/galaxy@280d2e3#diff-849c0107be4d923c00577caf4f84a1c5R38

[natefoo]

@ic4f yeah that was part of the 4-year-old installable Galaxy PR.

[jdavcs]

I cannot reproduce the xUnit error anymore (running on a Mac, Planemo runs on python 2.7.10 and 3.7.3; Galaxy on 2.7.10, dev branch. I tried both existing installs (with --galaxy_root) and a clean install; in all cases I did not switch branches). I only see a bad file descriptor error, which is not really a thing (discussed here: #752).

On a side note, the twobit.loc error (#679), which was present, has disappeared too.

@jmchilton I don't know Planemo well enough to dig any deeper at this point, so I'm leaving this as is. Tests run (at least on my machine).

[peterjc]

Also failing on my MIRA repository, with a weekly cron test, see:

https://travis-ci.org/peterjc/galaxy_mira/builds

e.g. 2 months ago passed:

https://travis-ci.org/peterjc/galaxy_mira/builds/552876215

The next week it failed:

https://travis-ci.org/peterjc/galaxy_mira/builds/555897289

There doesn't seem to be an easy way to get the date from a TravisCI build...