Skip to content

Commit

Permalink
Merge branch 'master' into maintenance/thinc-8.2.2-website
Browse files Browse the repository at this point in the history
  • Loading branch information
danieldk committed Dec 14, 2023
2 parents 62359a7 + b183006 commit afd164b
Show file tree
Hide file tree
Showing 181 changed files with 2,368 additions and 1,018 deletions.
44 changes: 0 additions & 44 deletions .github/workflows/autoblack.yml

This file was deleted.

4 changes: 2 additions & 2 deletions .github/workflows/issue-manager.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ jobs:
issue-manager:
runs-on: ubuntu-latest
steps:
- uses: tiangolo/issue-manager@0.2.1
- uses: tiangolo/issue-manager@0.4.0
with:
token: ${{ secrets.GITHUB_TOKEN }}
config: >
Expand All @@ -26,4 +26,4 @@ jobs:
"remove_label_on_comment": true,
"remove_label_on_close": true
}
}
}
163 changes: 163 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
name: tests

on:
push:
paths-ignore:
- "website/**"
- "*.md"
pull_request:
types: [opened, synchronize, reopened, edited]
paths-ignore:
- "website/**"
- "*.md"

jobs:
validate:
name: Validate
if: github.repository_owner == 'explosion'
runs-on: ubuntu-latest
steps:
- name: Check out repo
uses: actions/checkout@v3

- name: Configure Python version
uses: actions/setup-python@v4
with:
python-version: "3.7"
architecture: x64

- name: black
run: |
python -m pip install black -c requirements.txt
python -m black thinc --check
- name: isort
run: |
python -m pip install isort -c requirements.txt
python -m isort thinc --check
- name: flake8
run: |
python -m pip install flake8==5.0.4
python -m flake8 thinc --count --select=E901,E999,F821,F822,F823,W605 --show-source --statistics
tests:
name: Test
needs: Validate
if: github.repository_owner == 'explosion'
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
python_version: ["3.12"]
include:
- os: windows-2019
python_version: "3.6"
- os: macos-latest
python_version: "3.7"
- os: ubuntu-latest
python_version: "3.8"
- os: windows-latest
python_version: "3.9"
- os: macos-latest
python_version: "3.10"
- os: ubuntu-latest
python_version: "3.11"

runs-on: ${{ matrix.os }}
env:
NOTEBOOK_KERNEL: "thinc-notebook-tests"

steps:
- name: Check out repo
uses: actions/checkout@v3

- name: Configure Python version
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python_version }}
architecture: x64

- name: Install dependencies
run: |
python -m pip install --upgrade pip setuptools wheel
pip install -r requirements.txt
- name: Build sdist
run: |
python setup.py build_ext --inplace
python setup.py sdist --formats=gztar
- name: Run mypy
run: python -m mypy thinc --no-implicit-reexport
if: matrix.python_version != '3.6'

- name: Delete source directory
run: rm -rf thinc
shell: bash

- name: Uninstall all packages
run: |
python -m pip freeze
pip freeze --exclude pywin32 > installed.txt
pip uninstall -y -r installed.txt
- name: Install from sdist
run: |
SDIST=$(python -c "import os;print(os.listdir('./dist')[-1])" 2>&1)
PIP_CONSTRAINT="build-constraints.txt" pip install dist/$SDIST
shell: bash

- name: Test import
run: python -c "import thinc"

- name: Install test requirements
run: |
pip install -r requirements.txt
- name: Install notebook test requirements
run: |
pip install ipykernel pydot graphviz
python -m ipykernel install --name thinc-notebook-tests --user
if: matrix.python_version != '3.12'

- name: Run tests without extras
run: |
python -m pytest --pyargs thinc -Werror --cov=thinc --cov-report=term
# Notes on numpy requirements hacks:
# 1. torch does not have a direct numpy requirement but is compiled
# against a newer version than the oldest supported numpy for windows and
# python 3.10; this version of numpy would not work with
# tensorflow~=2.5.0 as specified above, but there is no release for
# python 3.10 anyway
# 2. restrict to numpy<1.24.0 due to mxnet incompatibility
# 3. forbid torch!=1.13.0 due to segfaults with numpy<1.24.0
# Note: some of these pip install commands are known to fail for some platforms.
# To continue despite errors as in azure pipelines, remove -e from the default
# bash flags.
- name: Install extras for testing
run: |
pip install "protobuf~=3.20.0" "tensorflow~=2.5.0"
pip install "mxnet; sys_platform != 'win32' and python_version < '3.12'"
pip install "torch!=1.13.0; sys_platform!='darwin'" --extra-index-url https://download.pytorch.org/whl/cpu
# there is a bug related to MPS devices in github macos runners that
# will be fixed in torch v2.1.1
# https://github.com/pytorch/pytorch/pull/111576
pip install "torch>=2.1.1; sys_platform=='darwin'" --extra-index-url https://download.pytorch.org/whl/cpu
pip install "numpy~=1.23.0; python_version=='3.10' and sys_platform=='win32'"
pip install "numpy<1.24.0"
pip install -r requirements.txt
pip uninstall -y mypy
shell: bash --noprofile --norc -o pipefail {0}

- name: Run tests with extras
run: python -m pytest --pyargs thinc --cov=thinc --cov-report=term -p thinc.tests.enable_tensorflow -p thinc.tests.enable_mxnet

- name: Run tests for thinc-apple-ops
run: |
pip uninstall -y tensorflow
pip install thinc-apple-ops
python -m pytest --pyargs thinc_apple_ops
if: matrix.os == 'macos-latest' && matrix.python_version == '3.10'

- name: Run tests with thinc-apple-ops
run: python -m pytest --pyargs thinc
if: matrix.os == 'macos-latest' && matrix.python_version == '3.10'
77 changes: 51 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,17 @@

### From the makers of [spaCy](https://spacy.io) and [Prodigy](https://prodi.gy)

[Thinc](https://thinc.ai) is a **lightweight deep learning library** that offers an elegant,
type-checked, functional-programming API for **composing models**, with support
for layers defined in other frameworks such as **PyTorch, TensorFlow and MXNet**. You
can use Thinc as an interface layer, a standalone toolkit or a flexible way to
develop new models. Previous versions of Thinc have been running quietly in
production in thousands of companies, via both [spaCy](https://spacy.io) and
[Prodigy](https://prodi.gy). We wrote the new version to let users **compose,
configure and deploy custom models** built with their favorite framework.

[![Azure Pipelines](https://img.shields.io/azure-devops/build/explosion-ai/public/7/master.svg?logo=azure-pipelines&style=flat-square)](https://dev.azure.com/explosion-ai/public/_build?definitionId=7)
[Thinc](https://thinc.ai) is a **lightweight deep learning library** that offers
an elegant, type-checked, functional-programming API for **composing models**,
with support for layers defined in other frameworks such as **PyTorch,
TensorFlow and MXNet**. You can use Thinc as an interface layer, a standalone
toolkit or a flexible way to develop new models. Previous versions of Thinc have
been running quietly in production in thousands of companies, via both
[spaCy](https://spacy.io) and [Prodigy](https://prodi.gy). We wrote the new
version to let users **compose, configure and deploy custom models** built with
their favorite framework.

[![tests](https://github.com/explosion/thinc/actions/workflows/tests.yml/badge.svg)](https://github.com/explosion/thinc/actions/workflows/tests.yml)
[![Current Release Version](https://img.shields.io/github/v/release/explosion/thinc.svg?include_prereleases&sort=semver&style=flat-square&logo=github)](https://github.com/explosion/thinc/releases)
[![PyPi Version](https://img.shields.io/pypi/v/thinc.svg?include_prereleases&sort=semver&style=flat-square&logo=pypi&logoColor=white)](https://pypi.python.org/pypi/thinc)
[![conda Version](https://img.shields.io/conda/vn/conda-forge/thinc.svg?style=flat-square&logo=conda-forge&logoColor=white)](https://anaconda.org/conda-forge/thinc)
Expand All @@ -23,18 +24,20 @@ configure and deploy custom models** built with their favorite framework.

## 🔥 Features

- **Type-check** your model definitions with custom types and [`mypy`](https://mypy.readthedocs.io/en/latest/) plugin.
- **Type-check** your model definitions with custom types and
[`mypy`](https://mypy.readthedocs.io/en/latest/) plugin.
- Wrap **PyTorch**, **TensorFlow** and **MXNet** models for use in your network.
- Concise **functional-programming** approach to model definition, using composition rather than inheritance.
- Concise **functional-programming** approach to model definition, using
composition rather than inheritance.
- Optional custom infix notation via **operator overloading**.
- Integrated **config system** to describe trees of objects and hyperparameters.
- Choice of **extensible backends**.
- **[Read more &rarr;](https://thinc.ai/docs)**

## 🚀 Quickstart

Thinc is compatible with **Python 3.6+** and runs on **Linux**,
**macOS** and **Windows**. The latest releases with binary wheels are available from
Thinc is compatible with **Python 3.6+** and runs on **Linux**, **macOS** and
**Windows**. The latest releases with binary wheels are available from
[pip](https://pypi.python.org/pypi/thinc). Before you install Thinc and its
dependencies, make sure that your `pip`, `setuptools` and `wheel` are up to
date. For the most recent releases, pip 19.3 or newer is recommended.
Expand All @@ -44,15 +47,23 @@ pip install -U pip setuptools wheel
pip install thinc
```

See the [extended installation docs](https://thinc.ai/docs/install#extended) for details on optional dependencies for different backends and GPU. You might also want to [set up static type checking](https://thinc.ai/docs/install#type-checking) to take advantage of Thinc's type system.
See the [extended installation docs](https://thinc.ai/docs/install#extended) for
details on optional dependencies for different backends and GPU. You might also
want to
[set up static type checking](https://thinc.ai/docs/install#type-checking) to
take advantage of Thinc's type system.

> ⚠️ If you have installed PyTorch and you are using Python 3.7+, uninstall the
> package `dataclasses` with `pip uninstall dataclasses`, since it may have
> been installed by PyTorch and is incompatible with Python 3.7+.
> package `dataclasses` with `pip uninstall dataclasses`, since it may have been
> installed by PyTorch and is incompatible with Python 3.7+.
### 📓 Selected examples and notebooks

Also see the [`/examples`](examples) directory and [usage documentation](https://thinc.ai/docs) for more examples. Most examples are Jupyter notebooks – to launch them on [Google Colab](https://colab.research.google.com) (with GPU support!) click on the button next to the notebook name.
Also see the [`/examples`](examples) directory and
[usage documentation](https://thinc.ai/docs) for more examples. Most examples
are Jupyter notebooks – to launch them on
[Google Colab](https://colab.research.google.com) (with GPU support!) click on
the button next to the notebook name.

| Notebook | Description |
| --------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
Expand All @@ -63,15 +74,20 @@ Also see the [`/examples`](examples) directory and [usage documentation](https:/

**[View more &rarr;](examples)**

[colab]: https://gistcdn.githack.com/ines/dcf354aa71a7665ae19871d7fd14a4e0/raw/461fc1f61a7bc5860f943cd4b6bcfabb8c8906e7/colab-badge.svg
[colab]:
https://gistcdn.githack.com/ines/dcf354aa71a7665ae19871d7fd14a4e0/raw/461fc1f61a7bc5860f943cd4b6bcfabb8c8906e7/colab-badge.svg
[intro_to_thinc]: examples/00_intro_to_thinc.ipynb
[intro_to_thinc_colab]: https://colab.research.google.com/github/explosion/thinc/blob/master/examples/00_intro_to_thinc.ipynb
[intro_to_thinc_colab]:
https://colab.research.google.com/github/explosion/thinc/blob/master/examples/00_intro_to_thinc.ipynb
[transformers_tagger_bert]: examples/02_transformers_tagger_bert.ipynb
[transformers_tagger_bert_colab]: https://colab.research.google.com/github/explosion/thinc/blob/master/examples/02_transformers_tagger_bert.ipynb
[transformers_tagger_bert_colab]:
https://colab.research.google.com/github/explosion/thinc/blob/master/examples/02_transformers_tagger_bert.ipynb
[pos_tagger_basic_cnn]: examples/03_pos_tagger_basic_cnn.ipynb
[pos_tagger_basic_cnn_colab]: https://colab.research.google.com/github/explosion/thinc/blob/master/examples/03_pos_tagger_basic_cnn.ipynb
[pos_tagger_basic_cnn_colab]:
https://colab.research.google.com/github/explosion/thinc/blob/master/examples/03_pos_tagger_basic_cnn.ipynb
[parallel_training_ray]: examples/04_parallel_training_ray.ipynb
[parallel_training_ray_colab]: https://colab.research.google.com/github/explosion/thinc/blob/master/examples/04_parallel_training_ray.ipynb
[parallel_training_ray_colab]:
https://colab.research.google.com/github/explosion/thinc/blob/master/examples/04_parallel_training_ray.ipynb

### 📖 Documentation & usage guides

Expand Down Expand Up @@ -103,7 +119,12 @@ Also see the [`/examples`](examples) directory and [usage documentation](https:/

## 🐍 Development notes

Thinc uses [`black`](https://github.com/psf/black) for auto-formatting, [`flake8`](http://flake8.pycqa.org/en/latest/) for linting and [`mypy`](https://mypy.readthedocs.io/en/latest/) for type checking. All code is written compatible with **Python 3.6+**, with type hints wherever possible. See the [type reference](https://thinc.ai/docs/api-types) for more details on Thinc's custom types.
Thinc uses [`black`](https://github.com/psf/black) for auto-formatting,
[`flake8`](http://flake8.pycqa.org/en/latest/) for linting and
[`mypy`](https://mypy.readthedocs.io/en/latest/) for type checking. All code is
written compatible with **Python 3.6+**, with type hints wherever possible. See
the [type reference](https://thinc.ai/docs/api-types) for more details on
Thinc's custom types.

### 👷‍♀️ Building Thinc from source

Expand Down Expand Up @@ -138,12 +159,16 @@ python setup.py build_ext --inplace

### 🚦 Running tests

Thinc comes with an [extensive test suite](thinc/tests). The following should all pass and not report any warnings or errors:
Thinc comes with an [extensive test suite](thinc/tests). The following should
all pass and not report any warnings or errors:

```bash
python -m pytest thinc # test suite
python -m mypy thinc # type checks
python -m flake8 thinc # linting
```

To view test coverage, you can run `python -m pytest thinc --cov=thinc`. We aim for a 100% test coverage. This doesn't mean that we meticulously write tests for every single line – we ignore blocks that are not relevant or difficult to test and make sure that the tests execute all code paths.
To view test coverage, you can run `python -m pytest thinc --cov=thinc`. We aim
for a 100% test coverage. This doesn't mean that we meticulously write tests for
every single line – we ignore blocks that are not relevant or difficult to test
and make sure that the tests execute all code paths.
Loading

0 comments on commit afd164b

Please sign in to comment.