Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensured Datasets Are Bundled Properly #14

Closed
AFg6K7h4fhy2 opened this issue Oct 9, 2024 · 1 comment
Closed

Ensured Datasets Are Bundled Properly #14

AFg6K7h4fhy2 opened this issue Oct 9, 2024 · 1 comment
Assignees
Labels
Low Priority A task that is of lower relative priority. testing Verifying something works as intended.

Comments

@AFg6K7h4fhy2
Copy link
Collaborator

These datasets include

  • location_data.csv
  • example_forecast.nc (NETCDF)
  • nhsn_fit_COVID.csv
  • nhsn_fit_flu.csv
  • example_flusight_submission.csv
@AFg6K7h4fhy2 AFg6K7h4fhy2 self-assigned this Oct 9, 2024
@AFg6K7h4fhy2 AFg6K7h4fhy2 added the documentation Explanations for aspects of the project. label Oct 9, 2024
@AFg6K7h4fhy2 AFg6K7h4fhy2 added testing Verifying something works as intended. and removed documentation Explanations for aspects of the project. labels Oct 21, 2024
@AFg6K7h4fhy2 AFg6K7h4fhy2 added the Low Priority A task that is of lower relative priority. label Oct 28, 2024
@AFg6K7h4fhy2
Copy link
Collaborator Author

As forecasttools stands now, the following data objects are contained:

  • example_flu_forecast_wo_dates.nc
  • example_flu_forecast_w_dates.nc
  • example_flusight_submission.parquet
  • location_table.parquet
  • nhsn_hosp_COVID.parquet
  • nhsn_hosp_flu.parquet

When an individual wants to use forecasttools via

pip3 install git+https://github.com/CDCgov/forecasttools-py@main

then the individual should expect to have accessible these data objects.

The code in pyproject.toml ensures the data objects are bundled in source distribution (sdist) and wheel (.whl) formats:

packages = [{include = "forecasttools"}]
include = [
    { path = "forecasttools/location_table.parquet", format = "sdist" },
    { path = "forecasttools/location_table.parquet", format = "wheel" },
    { path = "forecasttools/example_flusight_submission.parquet", format = "sdist" },
    { path = "forecasttools/example_flusight_submission.parquet", format = "wheel" },
    { path = "forecasttools/example_flu_forecast_wo_dates.nc", format = "sdist" },
    { path = "forecasttools/example_flu_forecast_wo_dates.nc", format = "wheel" },
    { path = "forecasttools/example_flu_forecast_w_dates.nc", format = "sdist" },
    { path = "forecasttools/example_flu_forecast_w_dates.nc", format = "wheel" },
    { path = "forecasttools/nhsn_hosp_COVID.parquet", format = "sdist" },
    { path = "forecasttools/nhsn_hosp_COVID.parquet", format = "wheel" },
    { path = "forecasttools/nhsn_hosp_flu.parquet", format = "sdist" },
    { path = "forecasttools/nhsn_hosp_flu.parquet", format = "wheel" },
]

The following can be used to test whether the data objects are properly packaged:

  1. Create an empty virtual environment, then activate it.
python3 -m venv data_bundling_test_env
source data_bundling_test_env/bin/activate
  1. Install forecasttools in the virtual environment.
pip3 install git+https://github.com/CDCgov/forecasttools-py@main

The following was outputted:

Collecting git+https://github.com/CDCgov/forecasttools-py@main
  Cloning https://github.com/CDCgov/forecasttools-py (to revision main) to /private/var/folders/cm/dlbpxkls2djcc1psjg2fzstc0000gn/T/pip-req-build-c0sxofrs
  Running command git clone --filter=blob:none --quiet https://github.com/CDCgov/forecasttools-py /private/var/folders/cm/dlbpxkls2djcc1psjg2fzstc0000gn/T/pip-req-build-c0sxofrs
  Resolved https://github.com/CDCgov/forecasttools-py to commit 4de2b4139c7200466094c5cfe4c9375a1a0fe104
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting arviz<0.21.0,>=0.20.0 (from forecasttools==0.0.1)
  Downloading arviz-0.20.0-py3-none-any.whl.metadata (8.8 kB)
Collecting epiweeks<3.0.0,>=2.3.0 (from forecasttools==0.0.1)
  Using cached epiweeks-2.3.0-py3-none-any.whl.metadata (2.6 kB)
Collecting matplotlib<4.0.0,>=3.9.2 (from forecasttools==0.0.1)
  Using cached matplotlib-3.9.2-cp312-cp312-macosx_11_0_arm64.whl.metadata (11 kB)
Collecting polars<2.0.0,>=1.8.2 (from forecasttools==0.0.1)
  Downloading polars-1.12.0-cp39-abi3-macosx_11_0_arm64.whl.metadata (14 kB)
Collecting xarray<2025.0.0,>=2024.9.0 (from forecasttools==0.0.1)
  Downloading xarray-2024.10.0-py3-none-any.whl.metadata (11 kB)
Collecting setuptools>=60.0.0 (from arviz<0.21.0,>=0.20.0->forecasttools==0.0.1)
  Using cached setuptools-75.3.0-py3-none-any.whl.metadata (6.9 kB)
Collecting numpy>=1.23.0 (from arviz<0.21.0,>=0.20.0->forecasttools==0.0.1)
  Downloading numpy-2.1.3-cp312-cp312-macosx_11_0_arm64.whl.metadata (62 kB)
Collecting scipy>=1.9.0 (from arviz<0.21.0,>=0.20.0->forecasttools==0.0.1)
  Using cached scipy-1.14.1-cp312-cp312-macosx_12_0_arm64.whl.metadata (60 kB)
Requirement already satisfied: packaging in ./data_bundling_test_env/lib/python3.12/site-packages (from arviz<0.21.0,>=0.20.0->forecasttools==0.0.1) (24.1)
Collecting pandas>=1.5.0 (from arviz<0.21.0,>=0.20.0->forecasttools==0.0.1)
  Downloading pandas-2.2.3-cp312-cp312-macosx_11_0_arm64.whl.metadata (89 kB)
Collecting h5netcdf>=1.0.2 (from arviz<0.21.0,>=0.20.0->forecasttools==0.0.1)
  Downloading h5netcdf-1.4.0-py3-none-any.whl.metadata (13 kB)
Collecting typing-extensions>=4.1.0 (from arviz<0.21.0,>=0.20.0->forecasttools==0.0.1)
  Using cached typing_extensions-4.12.2-py3-none-any.whl.metadata (3.0 kB)
Collecting xarray-einstats>=0.3 (from arviz<0.21.0,>=0.20.0->forecasttools==0.0.1)
  Downloading xarray_einstats-0.8.0-py3-none-any.whl.metadata (5.8 kB)
Collecting contourpy>=1.0.1 (from matplotlib<4.0.0,>=3.9.2->forecasttools==0.0.1)
  Downloading contourpy-1.3.0-cp312-cp312-macosx_11_0_arm64.whl.metadata (5.4 kB)
Collecting cycler>=0.10 (from matplotlib<4.0.0,>=3.9.2->forecasttools==0.0.1)
  Using cached cycler-0.12.1-py3-none-any.whl.metadata (3.8 kB)
Collecting fonttools>=4.22.0 (from matplotlib<4.0.0,>=3.9.2->forecasttools==0.0.1)
  Downloading fonttools-4.54.1-cp312-cp312-macosx_11_0_arm64.whl.metadata (163 kB)
Collecting kiwisolver>=1.3.1 (from matplotlib<4.0.0,>=3.9.2->forecasttools==0.0.1)
  Downloading kiwisolver-1.4.7-cp312-cp312-macosx_11_0_arm64.whl.metadata (6.3 kB)
Collecting pillow>=8 (from matplotlib<4.0.0,>=3.9.2->forecasttools==0.0.1)
  Downloading pillow-11.0.0-cp312-cp312-macosx_11_0_arm64.whl.metadata (9.1 kB)
Collecting pyparsing>=2.3.1 (from matplotlib<4.0.0,>=3.9.2->forecasttools==0.0.1)
  Downloading pyparsing-3.2.0-py3-none-any.whl.metadata (5.0 kB)
Collecting python-dateutil>=2.7 (from matplotlib<4.0.0,>=3.9.2->forecasttools==0.0.1)
  Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB)
Collecting h5py (from h5netcdf>=1.0.2->arviz<0.21.0,>=0.20.0->forecasttools==0.0.1)
  Downloading h5py-3.12.1-cp312-cp312-macosx_11_0_arm64.whl.metadata (2.5 kB)
Collecting pytz>=2020.1 (from pandas>=1.5.0->arviz<0.21.0,>=0.20.0->forecasttools==0.0.1)
  Downloading pytz-2024.2-py2.py3-none-any.whl.metadata (22 kB)
Collecting tzdata>=2022.7 (from pandas>=1.5.0->arviz<0.21.0,>=0.20.0->forecasttools==0.0.1)
  Downloading tzdata-2024.2-py2.py3-none-any.whl.metadata (1.4 kB)
Collecting six>=1.5 (from python-dateutil>=2.7->matplotlib<4.0.0,>=3.9.2->forecasttools==0.0.1)
  Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB)
Downloading arviz-0.20.0-py3-none-any.whl (1.7 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 4.6 MB/s eta 0:00:00
Using cached epiweeks-2.3.0-py3-none-any.whl (5.7 kB)
Using cached matplotlib-3.9.2-cp312-cp312-macosx_11_0_arm64.whl (7.8 MB)
Downloading polars-1.12.0-cp39-abi3-macosx_11_0_arm64.whl (28.9 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 28.9/28.9 MB 16.9 MB/s eta 0:00:00
Downloading xarray-2024.10.0-py3-none-any.whl (1.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 20.0 MB/s eta 0:00:00
Downloading contourpy-1.3.0-cp312-cp312-macosx_11_0_arm64.whl (251 kB)
Using cached cycler-0.12.1-py3-none-any.whl (8.3 kB)
Downloading fonttools-4.54.1-cp312-cp312-macosx_11_0_arm64.whl (2.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.3/2.3 MB 28.2 MB/s eta 0:00:00
Downloading h5netcdf-1.4.0-py3-none-any.whl (50 kB)
Downloading kiwisolver-1.4.7-cp312-cp312-macosx_11_0_arm64.whl (63 kB)
Downloading numpy-2.1.3-cp312-cp312-macosx_11_0_arm64.whl (13.5 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.5/13.5 MB 26.0 MB/s eta 0:00:00
Downloading pandas-2.2.3-cp312-cp312-macosx_11_0_arm64.whl (11.4 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.4/11.4 MB 26.2 MB/s eta 0:00:00
Downloading pillow-11.0.0-cp312-cp312-macosx_11_0_arm64.whl (3.0 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.0/3.0 MB 1.6 MB/s eta 0:00:00
Downloading pyparsing-3.2.0-py3-none-any.whl (106 kB)
Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
Using cached scipy-1.14.1-cp312-cp312-macosx_12_0_arm64.whl (29.9 MB)
Using cached setuptools-75.3.0-py3-none-any.whl (1.3 MB)
Using cached typing_extensions-4.12.2-py3-none-any.whl (37 kB)
Downloading xarray_einstats-0.8.0-py3-none-any.whl (32 kB)
Downloading pytz-2024.2-py2.py3-none-any.whl (508 kB)
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Downloading tzdata-2024.2-py2.py3-none-any.whl (346 kB)
Downloading h5py-3.12.1-cp312-cp312-macosx_11_0_arm64.whl (2.9 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.9/2.9 MB 82.1 MB/s eta 0:00:00
Building wheels for collected packages: forecasttools
  Building wheel for forecasttools (pyproject.toml) ... done
  Created wheel for forecasttools: filename=forecasttools-0.0.1-py3-none-any.whl size=3869507 sha256=68c5e07f045a855de942150fbd220155f98686d1d738a07592bb5dad960b358e
  Stored in directory: /private/var/folders/cm/dlbpxkls2djcc1psjg2fzstc0000gn/T/pip-ephem-wheel-cache-faq93ygh/wheels/0e/26/44/7657c96db44897dd4745e06a9bacc4e4e01b0229bb5236a838
Successfully built forecasttools
Installing collected packages: pytz, tzdata, typing-extensions, six, setuptools, pyparsing, polars, pillow, numpy, kiwisolver, fonttools, epiweeks, cycler, scipy, python-dateutil, h5py, contourpy, pandas, matplotlib, h5netcdf, xarray, xarray-einstats, arviz, forecasttools
Successfully installed arviz-0.20.0 contourpy-1.3.0 cycler-0.12.1 epiweeks-2.3.0 fonttools-4.54.1 forecasttools-0.0.1 h5netcdf-1.4.0 h5py-3.12.1 kiwisolver-1.4.7 matplotlib-3.9.2 numpy-2.1.3 pandas-2.2.3 pillow-11.0.0 polars-1.12.0 pyparsing-3.2.0 python-dateutil-2.9.0.post0 pytz-2024.2 scipy-1.14.1 setuptools-75.3.0 six-1.16.0 typing-extensions-4.12.2 tzdata-2024.2 xarray-2024.10.0 xarray-einstats-0.8.0

[notice] A new release of pip is available: 24.2 -> 24.3.1
[notice] To update, run: pip install --upgrade pip
  1. Create and run a Python file with the following:
import forecasttools


print(forecasttools.location_table)
print(forecasttools.nhsn_hosp_flu)
print(forecasttools.nhsn_hosp_COVID)
print(forecasttools.nhsn_flu_forecast_wo_dates)
print(forecasttools.nhsn_flu_forecast_w_dates)
print(forecasttools.example_flusight_submission)
  1. Deactivate the virtual environment:
deactivate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Low Priority A task that is of lower relative priority. testing Verifying something works as intended.
Projects
None yet
Development

No branches or pull requests

1 participant