Tools with optional output dataset directories fail if the directory does not exist

[natefoo]

An example from samtools stats, which defines the output:

        <data name="output" format="tabular" label="${tool.name} on ${on_string}">
            <discover_datasets pattern="(?P&lt;designation&gt;.+)\.tab" ext="tabular" visible="true" directory="split" />
        </data>

But does not create the split directory unless the corresponding tool parameter is selected. But even if the user selects not to split output, the tool fails during the finish step with the following (from main):

Traceback (most recent call last):
  File "/cvmfs/main.galaxyproject.org/galaxy/lib/galaxy/jobs/runners/__init__.py", line 685, in finish_job
    self._finish_or_resubmit_job(job_state, stdout, stderr, exit_code)
  File "/cvmfs/main.galaxyproject.org/galaxy/lib/galaxy/jobs/runners/__init__.py", line 432, in _finish_or_resubmit_job
    job_state.job_wrapper.finish(stdout, stderr, exit_code, check_output_detected_state=check_output_detected_state)
  File "/cvmfs/main.galaxyproject.org/galaxy/lib/galaxy/jobs/__init__.py", line 1381, in finish
    'primary': self.tool.collect_primary_datasets(out_data, self.get_tool_provided_job_metadata(), tool_working_directory, input_ext, input_dbkey)
  File "/cvmfs/main.galaxyproject.org/galaxy/lib/galaxy/tools/__init__.py", line 1614, in collect_primary_datasets
    return output_collect.collect_primary_datasets(self, output, tool_provided_metadata, job_working_directory, input_ext, input_dbkey=input_dbkey)
  File "/cvmfs/main.galaxyproject.org/galaxy/lib/galaxy/tools/parameters/output_collect.py", line 368, in collect_primary_datasets
    for discovered_file in discover_files(name, tool_provided_metadata, dataset_collectors, job_working_directory, outdata):
  File "/cvmfs/main.galaxyproject.org/galaxy/lib/galaxy/tools/parameters/output_collect.py", line 486, in discover_files
    for (match, collector) in walk_over_file_collectors(extra_file_collectors, job_working_directory, matchable):
  File "/cvmfs/main.galaxyproject.org/galaxy/lib/galaxy/tools/parameters/output_collect.py", line 503, in walk_over_file_collectors
    for match in walk_over_extra_files(extra_file_collector.directory, extra_file_collector, job_working_directory, matchable):
  File "/cvmfs/main.galaxyproject.org/galaxy/lib/galaxy/tools/parameters/output_collect.py", line 515, in walk_over_extra_files
    for filename in os.listdir(directory):
OSError: [Errno 2] No such file or directory: '/galaxy-repl/main/jobdir/018/337/18337952/working/split'

I suspect this was caused by #5240. This should probably block 18.01.

[jennaj]

self-tag so I can help test once corrected (independent and the reported use-case sent to bugs)

[mvdbeek]

Done with #5635