Skip to content

Commit

Permalink
Merge pull request #158 from GavinHuttley/develop
Browse files Browse the repository at this point in the history
DEV: more project rename updates
  • Loading branch information
GavinHuttley authored Sep 2, 2024
2 parents 7ea2453 + c8ffdf1 commit 9d46a11
Show file tree
Hide file tree
Showing 38 changed files with 223 additions and 204 deletions.
6 changes: 3 additions & 3 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ jobs:
- name: Upload sdist and wheel
uses: actions/upload-artifact@v4
with:
name: elt-wheel-sdist
name: eti-wheel-sdist
path: |
./dist/*.whl
./dist/*.tar.gz
Expand All @@ -69,7 +69,7 @@ jobs:
- name: Download sdist and wheel
uses: actions/download-artifact@v4
with:
name: elt-wheel-sdist
name: eti-wheel-sdist
path: ./dist

- name: Publish package distributions to Test PyPI
Expand All @@ -89,7 +89,7 @@ jobs:
- name: Download sdist and wheel
uses: actions/download-artifact@v4
with:
name: elt-wheel-sdist
name: eti-wheel-sdist
path: ./dist

- name: Publish package distributions to PyPI
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/testing_develop.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ jobs:
pip install --upgrade nox
- name: "Run nox for ${{ matrix.python-version }}"
run: "nox -s test-${{ matrix.python-version }} -- --cov-report lcov:lcov-${{matrix.os}}-${{matrix.python-version}}.lcov --cov-report term --cov-append --cov ensembl_lite"
run: "nox -s test-${{ matrix.python-version }} -- --cov-report lcov:lcov-${{matrix.os}}-${{matrix.python-version}}.lcov --cov-report term --cov-append --cov ensembl_tui"

- name: Coveralls Parallel
uses: coverallsapp/github-action@v2
Expand Down
70 changes: 37 additions & 33 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,26 @@
[![CI](https://github.com/cogent3/EnsemblLite/actions/workflows/testing_develop.yml/badge.svg)](https://github.com/cogent3/EnsemblLite/actions/workflows/testing_develop.yml)
[![CodeQL](https://github.com/cogent3/EnsemblLite/actions/workflows/codeql.yml/badge.svg)](https://github.com/cogent3/EnsemblLite/actions/workflows/codeql.yml)
[![Coverage Status](https://coveralls.io/repos/github/cogent3/EnsemblLite/badge.svg?branch=develop)](https://coveralls.io/github/cogent3/EnsemblLite?branch=develop)
[![CI](https://github.com/cogent3/ensembl_tui/actions/workflows/testing_develop.yml/badge.svg)](https://github.com/cogent3/ensembl_tui/actions/workflows/testing_develop.yml)
[![CodeQL](https://github.com/cogent3/ensembl_tui/actions/workflows/codeql.yml/badge.svg)](https://github.com/cogent3/ensembl_tui/actions/workflows/codeql.yml)
[![Coverage Status](https://coveralls.io/repos/github/cogent3/ensembl_tui/badge.svg?branch=develop)](https://coveralls.io/github/cogent3/ensembl_tui?branch=develop)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)

# EnsemblLite
# ensembl-tui

EnsemblLite provides the `elt` command line application for obtaining a subset of the data provided by Ensembl which can then be queried locally. You can have multiple such subsets on your machine, each corresponding to a different selection of species and data types.
ensembl-tui provides the `eti` terminal application for obtaining a subset of the data provided by Ensembl which can then be queried locally. You can have multiple such subsets on your machine, each corresponding to a different selection of species and data types.

> **Warning**
> EnsemblLite is in a preliminary phase of development with a limited feature set and incomplete test coverage! Please validate results against the web version. If you discover errors, please post a [bug report](https://github.com/cogent3/EnsemblLite/issues).
> ensembl-tui is in a preliminary phase of development with a limited feature set and incomplete test coverage! Please validate results against the web version. If you discover errors, please post a [bug report](https://github.com/cogent3/ensembl_tui/issues).
## Installing the software

<details>
<summary>General user installation instructions</summary>

```
$ pip install ensembl-tui
```

</details>

<details>
<summary>Developer installation instructions</summary>
Fork the repo and clone your fork to your local machine. In the terminal, create either a python virtual environment or a new conda environment and activate it. In that virtual environment
Expand All @@ -26,17 +36,11 @@ EnsemblLite provides the `elt` command line application for obtaining a subset o
```
</details>

<details>
<summary>General user installation instructions</summary>

We have not yet released on pypi. We will provide instructions here for a Docker based installation soon!
</details>

## Resources required to subset Ensembl data

Ensembl hosts some very large data sets. You need to have a machine with sufficient disk space to store the data you want to download. At present we do not have support for predicting how much storage would be required for a given selection of species and data types. We advise you to experiment.
Ensembl hosts some very large data sets. You need to have a machine with sufficient disk space to store the data you want to download. At present we do not have support for predicting how much storage would be required for a given selection of species and data types. You will need to experiment.

Some commands can be run in parallel but have moderate memory requirements. If you have a machine with limited RAM, you may need to reduce the number of parallel processes. Again, we advise you to experiment.
Some commands can be run in parallel but have moderate memory requirements. If you have a machine with limited RAM, you may need to reduce the number of parallel processes. Again, run some experiments.

## Getting setup

Expand All @@ -47,17 +51,17 @@ Some commands can be run in parallel but have moderate memory requirements. If y

<!-- [[[cog
import cog
from ensembl_lite import cli
from ensembl_tui import cli
from click.testing import CliRunner
runner = CliRunner()
result = runner.invoke(cli.main, ["exportrc", "--help"])
help = result.output.replace("Usage: main", "Usage: elt")
help = result.output.replace("Usage: main", "Usage: eti")
cog.out(
"```\n{}\n```".format(help)
)
]]] -->
```
Usage: elt exportrc [OPTIONS]
Usage: eti exportrc [OPTIONS]
exports sample config and species table to the nominated path
Expand All @@ -69,7 +73,7 @@ Some commands can be run in parallel but have moderate memory requirements. If y
<!-- [[[end]]] -->

```shell
$ elt exportrc -o ~/Desktop/Outbox/ensembl_download
$ eti exportrc -o ~/Desktop/Outbox/ensembl_download
```
This command creates a `ensembl_download` download directory and writes two plain text files into it:

Expand All @@ -85,17 +89,17 @@ Some commands can be run in parallel but have moderate memory requirements. If y

<!-- [[[cog
import cog
from ensembl_lite import cli
from ensembl_tui import cli
from click.testing import CliRunner
runner = CliRunner()
result = runner.invoke(cli.main, ["download", "--help"])
help = result.output.replace("Usage: main", "Usage: elt")
help = result.output.replace("Usage: main", "Usage: eti")
cog.out(
"```\n{}\n```".format(help)
)
]]] -->
```
Usage: elt download [OPTIONS]
Usage: eti download [OPTIONS]
download data from Ensembl's ftp site
Expand All @@ -114,7 +118,7 @@ Some commands can be run in parallel but have moderate memory requirements. If y

```shell
$ cd to/directory/with/config.cfg
$ elt download -c config.cfg
$ eti download -c config.cfg
```

> **Note**
Expand All @@ -129,17 +133,17 @@ The download creates a new `.cfg` file inside the download directory. This file

<!-- [[[cog
import cog
from ensembl_lite import cli
from ensembl_tui import cli
from click.testing import CliRunner
runner = CliRunner()
result = runner.invoke(cli.main, ["install", "--help"])
help = result.output.replace("Usage: main", "Usage: elt")
help = result.output.replace("Usage: main", "Usage: eti")
cog.out(
"```\n{}\n```".format(help)
)
]]] -->
```
Usage: elt install [OPTIONS]
Usage: eti install [OPTIONS]
create the local representations of the data
Expand All @@ -158,7 +162,7 @@ The following command uses 2 CPUs and has been safe on systems with only 16GB of

```shell
$ cd to/directory/with/downloaded_data
$ elt install -d downloaded_data -np 2
$ eti install -d downloaded_data -np 2
```

</details>
Expand All @@ -168,17 +172,17 @@ $ elt install -d downloaded_data -np 2

<!-- [[[cog
import cog
from ensembl_lite import cli
from ensembl_tui import cli
from click.testing import CliRunner
runner = CliRunner()
result = runner.invoke(cli.main, ["installed", "--help"])
help = result.output.replace("Usage: main", "Usage: elt")
help = result.output.replace("Usage: main", "Usage: eti")
cog.out(
"```\n{}\n```".format(help)
)
]]] -->
```
Usage: elt installed [OPTIONS]
Usage: eti installed [OPTIONS]
show what is installed
Expand All @@ -198,21 +202,21 @@ We provide a conventional command line interface for querying the data with subc
<details>
<summary>The full list of subcommands</summary>

You can get help on individual subcommands by running `elt <subcommand>` in the terminal.
You can get help on individual subcommands by running `eti <subcommand>` in the terminal.

<!-- [[[cog
import cog
from ensembl_lite import cli
from ensembl_tui import cli
from click.testing import CliRunner
runner = CliRunner()
result = runner.invoke(cli.main)
help = result.output.replace("Usage: main", "Usage: elt")
help = result.output.replace("Usage: main", "Usage: eti")
cog.out(
"```\n{}\n```".format(help)
)
]]] -->
```
Usage: elt [OPTIONS] COMMAND [ARGS]...
Usage: eti [OPTIONS] COMMAND [ARGS]...
Tools for obtaining and interrogating subsets of https://ensembl.org genomic
data.
Expand Down
23 changes: 10 additions & 13 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ requires = ["flit_core >=3.2,<4"]
build-backend = "flit_core.buildapi"

[project]
name = "ensembl_lite"
name = "ensembl_tui"
authors = [
{ name = "Gavin Huttley", email = "[email protected]"},
]
Expand Down Expand Up @@ -40,20 +40,20 @@ classifiers = [
dynamic = ["version", "description"]

[project.urls]
Documentation = "https://github.com/cogent3/EnsemblLite"
"Bug Tracker" = "https://github.com/cogent3/EnsemblLite/issues"
"Source Code" = "https://github.com/cogent3/EnsemblLite"
Documentation = "https://github.com/cogent3/ensembl_tui"
"Bug Tracker" = "https://github.com/cogent3/ensembl_tui/issues"
"Source Code" = "https://github.com/cogent3/ensembl_tui"

[project.scripts]
elt = "ensembl_lite.cli:main"
eti = "ensembl_tui.cli:main"

[project.optional-dependencies]
test = [
"click",
"pytest",
"pytest-cov",
"pytest-xdist",
"ruff==0.5.7",
"ruff==0.6.3",
"nox"]
doc = ["click==8.1.3",
"sphinx",
Expand Down Expand Up @@ -85,7 +85,7 @@ dev = ["click",
"pytest",
"pytest-cov",
"pytest-xdist",
"ruff==0.5.7",
"ruff==0.6.3",
"scriv",
"sphinx",
"sphinx-autobuild",
Expand All @@ -94,7 +94,7 @@ dev = ["click",
"sphinxcontrib-bibtex"]

[tool.flit.sdist]
include = ["src/*", "tests/", "pyproject.toml"]
include = ["src/*", "tests/*", "pyproject.toml"]

[tool.pytest.ini_options]
markers = [
Expand All @@ -107,7 +107,6 @@ testpaths = "tests"

[tool.ruff]
exclude = [
".bzr",
".direnv",
".eggs",
".git",
Expand All @@ -127,10 +126,8 @@ exclude = [
".vscode",
"__pypackages__",
"_build",
"buck-out",
"build",
"dist",
"node_modules",
"site-packages",
"venv",
]
Expand All @@ -139,7 +136,7 @@ exclude = [
line-length = 88
indent-width = 4

target-version = "py39"
target-version = "py310"

[tool.ruff.lint]
# Enable Pyflakes (`F`) and a subset of the pycodestyle (`E`) codes by default.
Expand Down Expand Up @@ -196,7 +193,7 @@ docstring-code-line-length = "dynamic"
format="md"
categories=["Contributors", "ENH", "BUG", "DOC", "Deprecations", "Discontinued"]
output_file="changelog.md"
version="literal: src/cogent3/__init__.py:__version__"
version="literal: src/ensembl_tui/__init__.py:__version__"
skip_fragments="README.*"
new_fragment_template="file: changelog.d/templates/new.md.j2"
entry_title_template="file: changelog.d/templates/title.md.j2"
File renamed without changes.
12 changes: 6 additions & 6 deletions src/ensembl_lite/_align.py → src/ensembl_tui/_align.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@
from cogent3.core.alignment import Aligned, Alignment
from cogent3.core.location import _DEFAULT_GAP_DTYPE, IndelMap

from ensembl_lite import _genome as elt_genome
from ensembl_lite import _storage_mixin as elt_mixin
from ensembl_lite import _util as elt_util
from ensembl_tui import _genome as elt_genome
from ensembl_tui import _storage_mixin as elt_mixin
from ensembl_tui import _util as elt_util

_no_gaps = numpy.array([], dtype=_DEFAULT_GAP_DTYPE)

Expand Down Expand Up @@ -81,7 +81,7 @@ class GapStore(elt_mixin.Hdf5Mixin):
def __init__(
self,
source: elt_util.PathType,
align_name: typing.Optional[str] = None,
align_name: str | None = None,
mode: str = "r",
in_memory: bool = False,
):
Expand Down Expand Up @@ -402,7 +402,7 @@ def _add_alignments(*alns, sep="?") -> Alignment:
all_names.update(set(aln.names))

result = {n: [] for n in all_names}
for aln, default in zip(alns, defaults):
for aln, default in zip(alns, defaults, strict=False):
data = aln.to_dict()
for name in all_names:
result[name].append(data.get(name, default))
Expand All @@ -423,7 +423,7 @@ def __init__(
self,
align_db: AlignDb,
genomes: dict[str, elt_genome.Genome],
mask_features: typing.Optional[list[str]] = None,
mask_features: list[str] | None = None,
sep: str = "?",
) -> None:
self._align_db = align_db
Expand Down
13 changes: 7 additions & 6 deletions src/ensembl_lite/_config.py → src/ensembl_tui/_config.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,13 @@
import configparser
import fnmatch
import pathlib
import typing
from collections.abc import Iterable
from dataclasses import dataclass

import click

from ensembl_lite import _species as elt_species
from ensembl_lite import _util as elt_util
from ensembl_tui import _species as elt_species
from ensembl_tui import _util as elt_util

INSTALLED_CONFIG_NAME = "installed.cfg"
DOWNLOADED_CONFIG_NAME = "downloaded.cfg"
Expand All @@ -25,7 +24,9 @@ def make_relative_to(
) -> pathlib.Path:
assert staging_path.is_absolute() and install_path.is_absolute()

for i, (s_part, i_part) in enumerate(zip(staging_path.parts, install_path.parts)):
for i, (s_part, i_part) in enumerate(
zip(staging_path.parts, install_path.parts, strict=False),
):
if s_part != i_part:
break
change_up = ("..",) * (len(staging_path.parts) - i)
Expand Down Expand Up @@ -243,11 +244,11 @@ def _standardise_path(

def read_config(
config_path: pathlib.Path,
root_dir: typing.Optional[pathlib.Path] = None,
root_dir: pathlib.Path | None = None,
) -> Config:
"""returns ensembl release, local path, and db specifics from the provided
config path"""
from ensembl_lite._download import download_ensembl_tree
from ensembl_tui._download import download_ensembl_tree

if not config_path.exists():
click.secho(f"File not found {config_path.resolve()!s}", fg="red")
Expand Down
Loading

0 comments on commit 9d46a11

Please sign in to comment.