GFF3 with CSI index shows "Error: Not a TBI file" in linear genome view

[laceysanderson]

Describe the bug

I've tried multiple GFF3 files which require CSI index and all of them show the same behaviour. Specifically, the track appears to be added to the configuration correctly as shown below but when I turn it on through the Linear Genome view track selector I see the following error message:

Here is the config file:

{
  "assemblies": [
    {
      "name": "Pisum sativum Cameor v1",
      "sequence": {
        "type": "ReferenceSequenceTrack",
        "trackId": "Pisum sativum Cameor v1-ReferenceSequenceTrack",
        "adapter": {
          "type": "BgzipFastaAdapter",
          "fastaLocation": {
            "uri": "https://v1.legumefederation.org/data/public/Pisum_sativum/Cameor.gnm1.P4FG/pissa.Cameor.gnm1.P4FG.genome_main.fna.gz"
          },
          "faiLocation": {
            "uri": "https://v1.legumefederation.org/data/public/Pisum_sativum/Cameor.gnm1.P4FG/pissa.Cameor.gnm1.P4FG.genome_main.fna.gz.fai"
          },
          "gziLocation": {
            "uri": "https://v1.legumefederation.org/data/public/Pisum_sativum/Cameor.gnm1.P4FG/pissa.Cameor.gnm1.P4FG.genome_main.fna.gz.gzi"
          }
        }
      },
      "aliases": [
        "pissa.Cameor"
      ]
    }
  ],
  "configuration": {},
  "connections": [],
  "defaultSession": {
    "name": "New Session"
  },
  "tracks": [
    {
      "type": "BasicTrack",
      "trackId": "genes",
      "name": "Genes",
      "category": [
        "Genes"
      ],
      "assemblyNames": [
        "Pisum sativum Cameor v1"
      ],
      "adapter": {
        "type": "Gff3TabixAdapter",
        "gffGzLocation": {
          "uri": "https://v1.legumefederation.org/data/public/Pisum_sativum/Cameor.gnm1.ann1.7SZR/pissa.Cameor.gnm1.ann1.7SZR.gene_models_main.gff3.gz"
        },
        "index": {
          "location": {
            "uri": "https://v1.legumefederation.org/data/public/Pisum_sativum/Cameor.gnm1.ann1.7SZR/pissa.Cameor.gnm1.ann1.7SZR.gene_models_main.gff3.gz.csi"
          },
          "indexType": "CSI"
        }
      }
    }
  ]
}

To Reproduce

1. Execute the following commands in your web root:

jbrowse create jbrowse2
cd jbrowse2
jbrowse add-assembly https://v1.legumefederation.org/data/public/Pisum_sativum/Cameor.gnm1.P4FG/pissa.Cameor.gnm1.P4FG.genome_main.fna.gz --name "Pisum sativum Cameor v1" --alias pissa.Cameor --type bgzipFasta
jbrowse add-track https://v1.legumefederation.org/data/public/Pisum_sativum/Cameor.gnm1.ann1.7SZR/pissa.Cameor.gnm1.ann1.7SZR.gene_models_main.gff3.gz.csi --name="Genes" --trackId=genes --category="Genes"

2. Navigate to http://localhost/jbrowse2.
3. Add a linear Genome View (File > Add > Linear Genome View)
4. Click the "Select Tracks" button and enable the "Genes" track. It will add the track but the error shown in the above screenshot should be visible instead of the gene glyphs.

NOTE: I'm using a docker image for easy reproducibility. You can access it here including instructions on it's usage. If using the docker, execute the following commands after starting your container rather then those listed in step 1 above.

docker exec useJBrowse jbrowse add-assembly https://v1.legumefederation.org/data/public/Pisum_sativum/Cameor.gnm1.P4FG/pissa.Cameor.gnm1.P4FG.genome_main.fna.gz --name "Pisum sativum Cameor v1" --alias pissa.Cameor --type bgzipFasta
docker exec useJBrowse jbrowse add-track https://v1.legumefederation.org/data/public/Pisum_sativum/Cameor.gnm1.ann1.7SZR/pissa.Cameor.gnm1.ann1.7SZR.gene_models_main.gff3.gz.csi --name="Genes" --trackId=genes --category="Genes"

I've also tested this on my debian web server with the same results.

Expected behavior

I expect to see the gene glyphs since #797 implies CSI is supported for GFF3 files (confirmed by config being correctly added).

Version:

Browser: Firefox 77.0.1 and Chrome 83.0.4103.116

$ jbrowse --version
@gmod/jbrowse-cli/0.0.1-beta.16 linux-x64 node-v10.21.0
$ node --version
v10.21.0
$ npm --version
6.14.4
$ cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

Additional context

The public example files used are from Pisum sativum (field pea) Cameor and available for usage here: https://v1.legumefederation.org/data/public/Pisum_sativum/. I've also tested this with Lentil but those files are not yet public.

[cmdcolin]

Nice catch, this should be fixed here on master now bde37a2

[cmdcolin]

We could cut a new release if that is helpful for testing

Might also be good if we made something like jbrowse create --branch master...our continuous integration does upload the latest builds to s3

[cmdcolin]

I think we can use the URL argument of jbrowse create/upgrade also

@laceysanderson if you're interested you could try upgrade/create with this

jbrowse upgrade --url https://jbrowse.org/code/jb2/alpha/master/jbrowse-web-master.zip

This will upgrade to a master branch build with the fix :)

[laceysanderson]

Using JBrowse update with the URL provided an error:

$ jbrowse upgrade --url https://jbrowse.org/code/jb2/alpha/master/jbrowse-web-master.zip
(node:144376) ExperimentalWarning: The fs.promises API is experimental
events.js:170
      throw er; // Unhandled 'error' event
      ^

Error: EISDIR: illegal operation on a directory, open 'static/'
Emitted 'error' event at:
    at fs.open (internal/fs/streams.js:279:12)
    at FSReqCallback.args [as oncomplete] (fs.js:145:20)

However, I spun up a development container and tested it on master and confirmed the commit does fix the error:

Thanks @cmdcolin!

[cmdcolin]

Super, glad that works

I might also recommend using the SvgFeatureRenderer. I used this command to load a GFF track before

jbrowse add-track https://s3.amazonaws.com/jbrowse.org/genomes/hg19/GRCh37_latest_genomic.sort.gff.gz --config '{"renderer": {"type": "SvgFeatureRenderer"}}'

With regards to that error running jbrowse upgrade with this, we will try and fix that too. I would love to use that technique in future issues if people are trying to debug their latest instance, hopefully we can get that upgrade path smoothed out. We saw that EISDIR error in an older version of our CLI but if it is still happening we will check into it

[cmdcolin]

I can confirm I have locally beta 0.0.16 and you also have 0.0.16 of the cli tools so it would appear it still happens under some circumstances