Skip to content
This repository has been archived by the owner on Apr 20, 2024. It is now read-only.

Commit

Permalink
CI setup, fixes for linting, type checking, etc.
Browse files Browse the repository at this point in the history
  • Loading branch information
vreuter committed Apr 14, 2024
1 parent c028211 commit e8b10d1
Show file tree
Hide file tree
Showing 15 changed files with 427 additions and 139 deletions.
11 changes: 11 additions & 0 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
name: CI

on: [push, pull_request]

jobs:
tests:
uses: ./.github/workflows/tests.yaml
lint:
uses: ./.github/workflows/lint.yaml
format:
uses: ./.github/workflows/format.yaml
27 changes: 27 additions & 0 deletions .github/workflows/format.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
name: format

on: [workflow_call]

jobs:
format:
strategy:
fail-fast: false
matrix:
python-version: ["3.10"]
os: [ubuntu-22.04]
runs-on: ${{ matrix.os }}

steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Install project
run: pip install .[formatting]
- name: Run black
run: black . --check --diff --color
- name: Run codespell
run: codespell --enable-colors
- name: Run isort
run: isort looptrace_napari tests --check --diff --color
25 changes: 25 additions & 0 deletions .github/workflows/lint.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
name: lint

on: [workflow_call]

jobs:
lint:
strategy:
fail-fast: false
matrix:
python-version: ["3.10"]
os: [ubuntu-22.04]
runs-on: ${{ matrix.os }}

steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Install project
run: pip install .[linting]
- name: Run mypy on Python ${{ matrix.python-version }} on ${{ matrix.os }}
run: mypy
- name: Run pylint on Python ${{ matrix.python-version }} on ${{ matrix.os }}
run: pylint looptrace_napari
23 changes: 23 additions & 0 deletions .github/workflows/tests.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
name: Tests

on: [workflow_call]

jobs:
tests:
strategy:
fail-fast: false
matrix:
python-version: [ "3.10", "3.11", "3.12" ]
os: [ ubuntu-latest, macos-latest, windows-latest, ubuntu-20.04 ]
runs-on: ${{ matrix.os }}

steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }} on ${{ matrix.os }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Install project
run: python -m pip install .[test]
- name: Run unit tests on Python ${{ matrix.python-version }} on ${{ matrix.os }}
run: pytest tests -vv --cov=./ --cov-report=xml
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,6 @@ build/
*.pyc
.venv/
__pycache__/

# test-related
.coverage*
27 changes: 27 additions & 0 deletions docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,30 @@ By default, with `nix-shell`, you should have all the dependencies you'll need n
In other words, dependencies to do things like run tests, run linter(s), and run type checking should all be provided.
If you try and that's not the case, please check the [Issue Tracker](https://github.com/gerlichlab/looptrace-napari/issues).
If an issue's open for what you're experiencing, please upvote the initial description of the issue and/or comment on the ticket.

### Testing, formatting, and linting
Here's what corresponds approximately to what's run for CI through the project's [GitHub actions workflows](../.github/workflows/).

NB: Each of the following commands is to be run _from the project [root folder](../)_.

__Run test suite__ with coverage statistics
```console
pytest tests -vv --cov=.
```

__Run formatters__
```console
black .
codespell
isort looptrace_napari tests
```

__Run type checker__
```console
mypy looptrace_napari
```

__Run linter__
```console
pylint looptrace_napari
```
2 changes: 1 addition & 1 deletion docs/user_docs/locus-spots.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,6 @@ __Scrolling__
In version 0.2 of this plugin, there's an image plane for every z-slice, for every timepoint, for every ROI in the particular field of view.
This can makes scrolling time-consuming and will be simplified in a future release.

__Finding points (centers of Guassian fits to pixel volume)__
__Finding points (centers of Gaussian fits to pixel volume)__

In version 0.2 of this plugin, a point (blue or red indicator) can only be visible only in the z-slice closest to the z-coordinate of the the center of the Gaussian fit to a particular pixel volume. Generally, these will be more toward the middle of the z-stack rather than at either end, so focus scrolling mainly through the midrange of the z-axis.
7 changes: 6 additions & 1 deletion looptrace_napari/_docs.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,17 @@
"""Documentation-related helpers"""

from dataclasses import dataclass

from typing_extensions import Annotated

from .types import PathOrPaths


@dataclass(frozen=True, kw_only=True)
class CommonReaderDoc:
path_type = Annotated[PathOrPaths, "The file(s) or folder(s) for which to attempt to infer reader function"]
path_type = Annotated[
PathOrPaths,
"The file(s) or folder(s) for which to attempt to infer reader function",
]
returns = "A None if we can't read the given path(s), else a reader function"
notes = "https://napari.org/stable/plugins/guides.html#layer-data-tuples"
23 changes: 16 additions & 7 deletions looptrace_napari/geometry.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,17 @@
from abc import abstractmethod
from dataclasses import dataclass
from typing import Protocol

from numpydoc_decorator import doc # type: ignore[import-untyped]

# TODO: need Python >= 3.12
# See: https://github.com/gerlichlab/looptrace-napari/issues/6
#from typing import override

from numpydoc_decorator import doc
# from typing import override


class LocatableXY(Protocol):

"""Something that admits x- and y-coordinate."""

@abstractmethod
def get_x_coordinate(self) -> float:
raise NotImplementedError
Expand All @@ -21,6 +23,13 @@ def get_y_coordinate(self) -> float:
raise NotImplementedError


@doc(
summary="Bundle x and y position to create point in 2D space.",
parameters=dict(
x="Position in x",
y="Position in y",
),
)
@dataclass(kw_only=True, frozen=True)
class ImagePoint2D(LocatableXY):
x: float
Expand All @@ -31,14 +40,14 @@ def __post_init__(self) -> None:
raise TypeError(f"At least one coordinate isn't floating-point! {self}")
if any(c < 0 for c in [self.x, self.y]):
raise ValueError(f"At least one coordinate is negative! {self}")

# TODO: adopt @override once on Python >= 3.12
# See: https://github.com/gerlichlab/looptrace-napari/issues/6
def get_x_coordinate(self) -> float:
return self.x

# TODO: adopt @override once on Python >= 3.12
# See: https://github.com/gerlichlab/looptrace-napari/issues/6
#@override
# @override
def get_y_coordinate(self) -> float:
return self.y
84 changes: 50 additions & 34 deletions looptrace_napari/locus_specific_points_reader.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,24 +4,19 @@
import logging
import os
from pathlib import Path
from typing import (
Callable,
Literal,
Optional,
Tuple,
Union,
)
from numpydoc_decorator import doc
from typing import Callable, Literal, Optional, Tuple, Union

from numpydoc_decorator import doc # type: ignore[import-untyped]

from ._docs import CommonReaderDoc
from .types import (
CsvRow,
from .types import ( # pylint: disable=unused-import
CsvRow,
LayerParams,
PathLike,
PathLike,
PointRecord,
Timepoint,
TraceId,
)
)

__author__ = "Vince Reuter"
__credits__ = ["Vince Reuter"]
Expand All @@ -35,56 +30,71 @@


@doc(
summary="This is the main hook required by napari / napari plugins to provide a Reader plugin.",
summary=(
"This is the main hook required by napari / napari plugins to provide a Reader"
" plugin."
),
returns=CommonReaderDoc.returns,
notes=CommonReaderDoc.notes,
)
def get_reader(path: CommonReaderDoc.path_type) -> Optional[Callable[[PathLike], list[FullLayerData]]]:
static_params = {"size": 0.5, "edge_width": 0.05, "edge_width_is_relative": True, "n_dimensional": False}
def get_reader(
path: CommonReaderDoc.path_type,
) -> Optional[Callable[[PathLike], list[FullLayerData]]]:
static_params = {
"size": 0.5,
"edge_width": 0.05,
"edge_width_is_relative": True,
"n_dimensional": False,
}

def do_not_parse(why: str, *, level: int = logging.DEBUG):
logging.log(level=level, msg=f"{why}, cannot be read as looptrace locus-specific points: {path}")
return None
def do_not_parse(why: str, *, level: int = logging.DEBUG) -> None:
return logging.log(
level=level,
msg=f"{why}, cannot be read as looptrace locus-specific points: {path}",
)

# Input should be a single extant filepath.
if not isinstance(path, (str, Path)):
return do_not_parse("Not a path-like")
return do_not_parse("Not a path-like") # type: ignore[func-returns-value]
if not os.path.isfile(path):
return do_not_parse("Not an extant file")
return do_not_parse("Not an extant file") # type: ignore[func-returns-value]

# Input should also be a CSV.
base, ext = os.path.splitext(os.path.basename(path))
if ext != ".csv":
return do_not_parse("Not a CSV")
return do_not_parse("Not a CSV") # type: ignore[func-returns-value]

# Determine how to read and display the points layer to be parsed.
status_name = base.lower().split(".")[-1].lstrip("qc_").lstrip("qc")
if status_name in {"pass", "passed"}:
logging.debug(f"Parsing as QC-pass: {path}")
logging.debug("Parsing as QC-pass: %s", path)
color = "red"
symbol = "*"
read_rows = parse_passed
elif status_name in {"fail", "failed"}:
logging.debug(f"Parsing as QC-fail: {path}")
logging.debug("Parsing as QC-fail: %s", path)
color = "blue"
symbol = "o"
read_rows = parse_failed
else:
return do_not_parse(f"Could not infer QC status from '{status_name}'", level=logging.WARNING)

return do_not_parse( # type: ignore[func-returns-value]
f"Could not infer QC status from '{status_name}'", level=logging.WARNING
)

# Build the actual parser function.
base_meta = {"edge_color": color, "face_color": color, "symbol": symbol}

def parse(p):
with open(p, mode='r', newline='') as fh:
with open(p, mode="r", newline="") as fh:
rows = list(csv.reader(fh))
data, extra_meta = read_rows(rows)
if not data:
logging.warning("No data rows parsed!")
params = {**static_params, **base_meta, **extra_meta}
return data, params, "points"

return lambda p: [parse(p)]


@doc(
summary="Parse records from points which passed QC.",
Expand Down Expand Up @@ -113,7 +123,9 @@ def parse_failed(records: list[CsvRow]) -> Tuple[RawLocusPointsLike, LayerParams
data = []
codes = []
else:
data_codes_pairs: list[Tuple[PointRecord, QCFailReasons]] = [(parse_simple_record(r, exp_num_fields=6), r[5]) for r in records]
data_codes_pairs: list[Tuple[PointRecord, QCFailReasons]] = [
(parse_simple_record(r, exp_num_fields=6), r[5]) for r in records
]
data, codes = map(list, zip(*data_codes_pairs))
return data, {"text": QC_FAIL_CODES_KEY, "properties": {QC_FAIL_CODES_KEY: codes}}

Expand All @@ -122,19 +134,23 @@ def parse_failed(records: list[CsvRow]) -> Tuple[RawLocusPointsLike, LayerParams
summary="Parse single-point from a single record (e.g., row from a CSV file).",
parameters=dict(
r="Record (e.g. CSV row) to parse",
exp_num_fields="The expected number of data fields (e.g., columns) in the record",
exp_num_fields=(
"The expected number of data fields (e.g., columns) in the record"
),
),
returns="""
A pair of values in which the first element represents a locus spot's trace ID and timepoint,
and the second element represents the (z, y, x) coordinates of the centroid of the spot fit.
""",
)
def parse_simple_record(r: CsvRow, *, exp_num_fields: int) -> PointRecord:
def parse_simple_record(r: CsvRow, *, exp_num_fields: int) -> RawLocusPointsLike:
"""Parse a single line from an input CSV file."""
if not isinstance(r, list):
raise TypeError(f"Record to parse must be list, not {type(r).__name__}")
if len(r) != exp_num_fields:
raise ValueError(f"Expected record of length {exp_num_fields} but got {len(r)}: {r}")
raise ValueError(
f"Expected record of length {exp_num_fields} but got {len(r)}: {r}"
)
trace = int(r[0])
timepoint = int(r[1])
z = float(r[2])
Expand Down
Loading

0 comments on commit e8b10d1

Please sign in to comment.