Conda tutorial fixes. Enable --conda_use_local option for planemo test

docs/_writing_conda_new.rst

@@ -53,4 +53,4 @@ Exercise - Build a Recipe
 ----------------------------------------------------------------
 
 If you have just completed the exercise above - this exercise can be found in parent folder. Get
-there with ``cd ../exercise2``. If not, the exercise can be downloaded with
+there with ``cd ../exercise_2``. If not, the exercise can be downloaded with

docs/_writing_dependencies_conda.rst

@@ -200,8 +200,8 @@ Use the ``project_init`` command to download this exercise.
 ::
 
     $ planemo project_init --template conda_exercises conda_exercises
-    $ cd conda_exercises/exercise1
-    $ ls 
+    $ cd conda_exercises/exercise_1
+    $ ls
     pear.xml              test-data
 
 This project template contains a few exercises. The first uses an adapted
@@ -226,19 +226,18 @@ no ``requirement`` tags and so will not work properly.
 ::
 
     $ planemo project_init --template conda_exercises conda_exercises
-    $ cd conda_exercises/exercise2
-    $ ls 
+    $ cd conda_exercises/exercise_2
+    $ ls
     fleeqtk_seq.xml              test-data
 
 .. include:: _writing_conda_fleeqtk.rst
 
 1. Clone and branch Bioconda_.
-2. Build a recipe for fleeqtk version 1.3. You may wish to use ``conda skeleton``, start from
-   scratch, or copy the recipe of seqtk and work from there - any of these strategies should work.  
+2. Build a recipe for fleeqtk version 1.3. You may wish to start from scratch
+   (``conda skeleton`` is not available for C programs like fleeqtk), or copy
+   the recipe of seqtk and modify it for fleeqtk.
 3. Use ``conda build`` or Bioconda tooling to build the recipe.
-4. Run ``planemo conda_install --conda_use_local fleeqtk_seq.xml`` to verify the resulting package
-   can be built into a Galaxy environment.
-5. Run ``planemo test fleeqtk_seq.xml`` to verify the resulting package works as expected.
+4. Run ``planemo test --conda_use_local fleeqtk_seq.xml`` to verify the resulting package works as expected.
 
 .. note: The planemo flag ``--conda_use_local`` causes Planemo to use locally built
      packages during dependency resolution and related commands.

docs/_writing_dependencies_conda_cwl.rst

@@ -77,7 +77,7 @@ listed requirements are found.
 
 We can verify these tool requirements install with the ``conda_install`` command. With
 its default parameters ``conda_install`` processes tools and creates isolated environments
-for their declared `Software Requirements`_ (mirroring what can be done in production with 
+for their declared `Software Requirements`_ (mirroring what can be done in production with
 cwltool_ and Toil_).
 
 ::
@@ -178,7 +178,7 @@ demonstrating using this tool.
     All 1 test(s) executed passed.
     seqtk_seq_0: passed
 
-Since ``seqtk`` isn't on the path and we did not use a container, we can see the SoftwareRequirement 
+Since ``seqtk`` isn't on the path and we did not use a container, we can see the SoftwareRequirement
 resolution was successful and it found the environment we previously installed with ``conda_install``.
 
 This can be used outside of Planemo testing as well, the following invocation shows running a job
@@ -370,8 +370,8 @@ Use the ``project_init`` command to download this exercise.
 ::
 
     $ planemo project_init --template conda_exercises_cwl conda_exercises
-    $ cd conda_exercises/exercise1
-    $ ls 
+    $ cd conda_exercises/exercise_1
+    $ ls
     pear.cwl              test-data
 
 This project template contains a few exercises. The first uses a CWL tool for
@@ -397,19 +397,18 @@ not work properly without modification.
 ::
 
     $ planemo project_init --template conda_exercises_cwl conda_exercises
-    $ cd conda_exercises/exercise2
-    $ ls 
+    $ cd conda_exercises/exercise_2
+    $ ls
     fleeqtk_seq.cwl      fleeqtk_seq_tests.yml         test-data
 
 .. include:: _writing_conda_fleeqtk.rst
 
 1. Clone and branch Bioconda_.
-2. Build a recipe for fleeqtk version 1.3. You may wish to use ``conda skeleton``, start from
-   scratch, or copy the recipe of seqtk and work from there - any of these strategies should work.  
+2. Build a recipe for fleeqtk version 1.3. You may wish to start from scratch
+   (``conda skeleton`` is not available for C programs like fleeqtk), or copy
+   the recipe of seqtk and modify it for fleeqtk.
 3. Use ``conda build`` or Bioconda tooling to build the recipe.
-4. Run ``planemo conda_install --conda_use_local fleeqtk_seq.cwl`` to verify the resulting package
-   can be built into a Galaxy environment.
-5. Run ``planemo test fleeqtk_seq.cwl`` to verify the resulting package works as expected.
+4. Run ``planemo test --conda_use_local fleeqtk_seq.cwl`` to verify the resulting package works as expected.
 
 .. note: The planemo flag ``--conda_use_local`` causes Planemo to use locally built
      packages during dependency resolution and related commands.

docs/_writing_multiple_outputs.rst

@@ -166,7 +166,7 @@ Consider a tool that creates a bunch of text files or bam files and writes them
 .. code-block:: xml
 
     <outputs>
-        <data format="txt" name="report">
+        <data name="report" format="txt">
             <discover_datasets pattern="__designation_and_ext__" directory="split" visible="true" />
         </data>
     </outputs>
@@ -178,8 +178,8 @@ If the tool doesn't create the files in ``split`` with extensions or does but wi
 .. code-block:: xml
 
     <outputs>
-        <data format="txt" name="report">
-        <discover_datasets pattern="__designation__" format="tabular" directory="tables" visible="true" />
+        <data name="report" format="txt">
+            <discover_datasets pattern="__designation__" format="tabular" directory="tables" visible="true" />
         </data>
     </outputs>
 
@@ -190,8 +190,8 @@ It may not be desirable for the extension/format (``.tsv``) to appear in the ``d
 .. code-block:: xml
 
     <outputs>
-        <data format="txt" name="report">
-        <discover_datasets pattern="(?P&lt;designation&gt;.+)\.tsv" format="tabular" directory="tables" visible="true" />
+        <data name="report" format="txt">
+            <discover_datasets pattern="(?P&lt;designation&gt;.+)\.tsv" format="tabular" directory="tables" visible="true" />
         </data>
     </outputs>
 

docs/tests/tests_conda.sh

@@ -89,5 +89,4 @@ conda build fleeqtk
 # Update to fix tool and see it work.
 
 wget https://raw.githubusercontent.com/galaxyproject/planemo/master/project_templates/conda_answers/exercise_2/fleeqtk_seq.xml -O fleeqtk_seq.xml
-planemo conda_install --conda_use_local fleeqtk_seq.xml
-planemo test fleeqtk_seq.xml
+planemo test --conda_use_local fleeqtk_seq.xml

docs/tests/tests_conda_cwl.sh

@@ -83,5 +83,4 @@ conda build fleeqtk
 # Update to fix tool and see it work.
 
 wget https://raw.githubusercontent.com/galaxyproject/planemo/master/project_templates/conda_answers_cwl/exercise_2/fleeqtk_seq.cwl -O fleeqtk_seq.cwl
-planemo conda_install --conda_use_local fleeqtk_seq.cwl
-planemo test --no-container fleeqtk_seq.cwl
+planemo test --conda_use_local --no-container fleeqtk_seq.cwl

planemo/deps.py

@@ -64,6 +64,7 @@ def ensure_dependency_resolvers_conf_configured(ctx, kwds, resolvers_conf=None):
         'conda_auto_init': always_specify_attribute,
         'conda_auto_install': always_specify_attribute,
         'conda_ensure_channels': '',
+        'conda_use_local': False,
     }
     attributes = []
 

planemo/galaxy/config.py

@@ -399,7 +399,7 @@ def config_join(*args):
         )
         _ensure_directory(shed_tool_path)
         port = _get_port(kwds)
-        if parse_version(kwds.get('galaxy_python_version', DEFAULT_PYTHON_VERSION)) >= parse_version('3'):
+        if parse_version(kwds.get('galaxy_python_version') or DEFAULT_PYTHON_VERSION) >= parse_version('3'):
             # on python 3 we use gunicorn,
             # which requires 'main' as server name
             server_name = 'main'
@@ -947,7 +947,7 @@ def startup_command(self, ctx, **kwds):
             run_script += " --server-name %s" % shlex_quote(self.server_name)
         server_ini = os.path.join(self.config_directory, "galaxy.ini")
         self.env["GALAXY_CONFIG_FILE"] = server_ini
-        if parse_version(kwds.get('galaxy_python_version', DEFAULT_PYTHON_VERSION)) >= parse_version('3'):
+        if parse_version(kwds.get('galaxy_python_version') or DEFAULT_PYTHON_VERSION) >= parse_version('3'):
             # We need to start under gunicorn
             self.env['APP_WEBSERVER'] = 'gunicorn'
             self.env['GUNICORN_CMD_ARGS'] = "--bind={host}:{port} --name={server_name}".format(

planemo/galaxy/run.py

@@ -58,7 +58,7 @@ def locate_galaxy_virtualenv(ctx, kwds):
         workspace = ctx.workspace
         galaxy_branch = kwds.get("galaxy_branch") or "master"
         shared_venv_path = os.path.join(workspace, "gx_venv")
-        galaxy_python_version = kwds.get('galaxy_python_version', DEFAULT_PYTHON_VERSION)
+        galaxy_python_version = kwds.get('galaxy_python_version') or DEFAULT_PYTHON_VERSION
         if galaxy_python_version != DEFAULT_PYTHON_VERSION:
             shared_venv_path = "%s_%s" % (shared_venv_path, galaxy_python_version)
         if galaxy_branch != "master":

planemo/options.py

@@ -133,7 +133,7 @@ def galaxy_python_version():
     return planemo_option(
         '--galaxy_python_version',
         use_global_config=True,
-        default='2.7',
+        default=None,
         type=click.Choice(['2', '2.7', '3', '3.3', '3.4', '3.5', '3.6', '3.7']),
         help="Python version to start Galaxy under",
     )

project_templates/conda_exercises/exercise_2/fleeqtk_seq.xml

@@ -1,4 +1,7 @@
 <tool id="fleeqtk_seq" name="Convert to FASTA (fleeqtk)" version="0.1.0">
+    <requirements>
+        <requirement type="package" version="1.3">fleeqtk</requirement>
+    </requirements>
     <command detect_errors="exit_code"><![CDATA[
         fleeqtk seq
               #if $settings.advanced == "advanced"

project_templates/conda_exercises_cwl/exercise_2/fleeqtk_seq.cwl

@@ -3,6 +3,12 @@ cwlVersion: 'v1.0'
 class: CommandLineTool
 id: "fleeqtk_seq"
 label: "Convert to FASTA (fleeqtk)"
+hints:
+  SoftwareRequirement:
+    packages:
+    - package: fleeqtk
+      version:
+      - "1.3"
 inputs:
   input1:
     type: File

tests/test_cmd_training_generate_from_wf.py

@@ -74,7 +74,7 @@ def test_training_generate_from_wf_command_remote_wf(self):
                 "--tutorial_name", tuto_n,
                 "--workflow_id", "ID"
             ]
-            self._check_exit_code(training_init_command, exit_code=1)
+            self._check_exit_code(training_init_command, exit_code=self.non_zero_exit_code)
             # not working test
             create_tutorial_dir(topic_n, tuto_n)
             training_init_command = [
@@ -85,5 +85,5 @@ def test_training_generate_from_wf_command_remote_wf(self):
                 "--galaxy_url", "https://usegalaxy.eu/",
                 "--galaxy_api_key", "API"
             ]
-            self._check_exit_code(training_init_command, exit_code=1)
+            self._check_exit_code(training_init_command, exit_code=self.non_zero_exit_code)
             shutil.rmtree("topics")

tests/test_shed_lint.py

@@ -58,11 +58,12 @@ def test_invalid_nested(self):
 
     def test_fail_fast(self):
         # Created a nested repository with one good and one
-        # invalid repository and make sure it runs and produces
-        # a 254 (it ran to completion but one or more things failed
-        # )
-        with self._isolate_repo("bad_invalid_yaml"):
-            r = self._check_exit_code(["shed_lint", "--fail_fast"],
+        # invalid repository and make sure it exits immediately with 1.
+        with self._isolate() as f:
+            for name in ["bad_invalid_yaml", "single_tool_exclude"]:
+                self._copy_repo(name, join(f, name))
+                self._copy_repo(name, join(f, name))
+            r = self._check_exit_code(["shed_lint", "-r", "--fail_fast"],
                                       exit_code=1)
             assert isinstance(r.exception, RuntimeError)