Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reco utils to recommenders #1485

Merged
merged 4 commits into from
Jul 26, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion MANIFEST.in
Original file line number Diff line number Diff line change
@@ -1 +1 @@
include reco_utils/README.md
include recommenders/README.md
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ This repository contains examples and best practices for building recommendation
- [Model Select and Optimize](examples/04_model_select_and_optimize): Tuning and optimizing hyperparameters for recommender models
- [Operationalize](examples/05_operationalize): Operationalizing models in a production environment on Azure

Several utilities are provided in [reco_utils](reco_utils) to support common tasks such as loading datasets in the format expected by different algorithms, evaluating model outputs, and splitting training/test data. Implementations of several state-of-the-art algorithms are included for self-study and customization in your own applications. See the [reco_utils documentation](https://readthedocs.org/projects/microsoft-recommenders/).
Several utilities are provided in [recommenders](recommenders) to support common tasks such as loading datasets in the format expected by different algorithms, evaluating model outputs, and splitting training/test data. Implementations of several state-of-the-art algorithms are included for self-study and customization in your own applications. See the [recommenders documentation](https://readthedocs.org/projects/microsoft-recommenders/).

For a more detailed overview of the repository, please see the documents on the [wiki page](https://github.com/microsoft/recommenders/wiki/Documents-and-Presentations).

Expand Down Expand Up @@ -65,7 +65,7 @@ jupyter notebook

6. Run the [SAR Python CPU MovieLens](examples/00_quick_start/sar_movielens.ipynb) notebook under the `00_quick_start` folder. Make sure to change the kernel to "Python (reco)".

For additional options to install the package (support for GPU, Spark etc.) see [this guide](reco_utils/README.md).
For additional options to install the package (support for GPU, Spark etc.) see [this guide](recommenders/README.md).

**NOTE** - The [Alternating Least Squares (ALS)](examples/00_quick_start/als_movielens.ipynb) notebooks require a PySpark environment to run. Please follow the steps in the [setup guide](SETUP.md#dependencies-setup) to run these notebooks in a PySpark environment. For the deep learning algorithms, it is recommended to use a GPU machine and to follow the steps in the [setup guide](SETUP.md#dependencies-setup) to set up Nvidia libraries.

Expand Down Expand Up @@ -143,7 +143,7 @@ This project welcomes contributions and suggestions. Before contributing, please

## Build Status

These tests are the nightly builds, which compute the smoke and integration tests. `main` is our principal branch and `staging` is our development branch. We use `pytest` for testing python utilities in [reco_utils](reco_utils) and `papermill` for the [notebooks](examples). For more information about the testing pipelines, please see the [test documentation](tests/README.md).
These tests are the nightly builds, which compute the smoke and integration tests. `main` is our principal branch and `staging` is our development branch. We use `pytest` for testing python utilities in [recommenders](recommenders) and `papermill` for the [notebooks](examples). For more information about the testing pipelines, please see the [test documentation](tests/README.md).

### DSVM Build Status

Expand Down
8 changes: 4 additions & 4 deletions SETUP.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ conda update conda -n root
conda update anaconda # use 'conda install anaconda' if the package is not installed
```

There are different ways one may use the recommenders utilities. The most convenient one is probably by installing the `recommenders` package from [PyPI](https://pypi.org). For instructions on how to do these, see [this guide](reco_utils/README.md).
There are different ways one may use the recommenders utilities. The most convenient one is probably by installing the `recommenders` package from [PyPI](https://pypi.org). For instructions on how to do these, see [this guide](recommenders/README.md).

An alternative is to run all the recommender utilities directly from a local copy of the source code. This requires installing all the necessary dependencies from Anaconda and PyPI. For instructions on how to do this, see [this guide](conda.md)

Expand Down Expand Up @@ -307,12 +307,12 @@ To install the repo manually onto Databricks, follow the steps:
After installation, you can now create a new notebook and import the utilities from Databricks in order to confirm that the import worked.

```{python}
import reco_utils
import recommenders
```

### Troubleshooting Installation on Azure Databricks

* For the [reco_utils](reco_utils) import to work on Databricks, it is important to zip the content correctly. The zip has to be performed inside the Recommenders folder, if you zip directly above the Recommenders folder, it won't work.
* For the [recommenders](recommenders) import to work on Databricks, it is important to zip the content correctly. The zip has to be performed inside the Recommenders folder, if you zip directly above the Recommenders folder, it won't work.

### Prepare Azure Databricks for Operationalization

Expand Down Expand Up @@ -378,7 +378,7 @@ You can then open the Jupyter notebook server at http://localhost:8888

The process of making a new release and publishing it to pypi is as follows:

First make sure that the tag that you want to add, e.g. `0.6.0`, is added in [reco_utils.py/__init__.py](reco_utils.py/__init__.py). Follow the [contribution guideline](CONTRIBUTING.md) to add the change.
First make sure that the tag that you want to add, e.g. `0.6.0`, is added in [recommenders.py/__init__.py](recommenders.py/__init__.py). Follow the [contribution guideline](CONTRIBUTING.md) to add the change.

1. Make sure that the code in main passes all the tests (unit and nightly tests).
1. Create a tag with the version number: e.g. `git tag -a 0.6.0 -m "Recommenders 0.6.0"`.
Expand Down
2 changes: 1 addition & 1 deletion contrib/azureml_designer_modules/entries/map_entry.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
load_data_frame_from_directory,
save_data_frame_to_directory,
)
from reco_utils.evaluation.python_evaluation import map_at_k
from recommenders.evaluation.python_evaluation import map_at_k


if __name__ == "__main__":
Expand Down
2 changes: 1 addition & 1 deletion contrib/azureml_designer_modules/entries/ndcg_entry.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
load_data_frame_from_directory,
save_data_frame_to_directory,
)
from reco_utils.evaluation.python_evaluation import ndcg_at_k
from recommenders.evaluation.python_evaluation import ndcg_at_k


if __name__ == "__main__":
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
load_data_frame_from_directory,
save_data_frame_to_directory,
)
from reco_utils.evaluation.python_evaluation import precision_at_k
from recommenders.evaluation.python_evaluation import precision_at_k


if __name__ == "__main__":
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
load_data_frame_from_directory,
save_data_frame_to_directory,
)
from reco_utils.evaluation.python_evaluation import recall_at_k
from recommenders.evaluation.python_evaluation import recall_at_k


if __name__ == "__main__":
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import argparse

from azureml.studio.core.logger import module_logger as logger
from reco_utils.datasets.python_splitters import python_stratified_split
from recommenders.datasets.python_splitters import python_stratified_split
from azureml.studio.core.data_frame_schema import DataFrameSchema
from azureml.studio.core.io.data_frame_directory import (
load_data_frame_from_directory,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
import joblib
from pathlib import Path

from reco_utils.models.sar import SAR
from recommenders.models.sar import SAR

from azureml.studio.core.logger import module_logger as logger
from azureml.studio.core.utils.fileutils import ensure_folder
Expand Down
2 changes: 1 addition & 1 deletion docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
sys.path.insert(0, os.path.abspath(os.path.join("..", "..")))
sys.setrecursionlimit(1500)

from reco_utils import TITLE, VERSION, COPYRIGHT, AUTHOR
from recommenders import TITLE, VERSION, COPYRIGHT, AUTHOR

# -- Project information -----------------------------------------------------

Expand Down
26 changes: 13 additions & 13 deletions docs/source/datasets.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ information, ratings, and a plaintext review.
J. McAuley and J. Leskovec, "Hidden factors and hidden topics: understanding rating dimensions with review text",
RecSys, 2013.

.. automodule:: reco_utils.datasets.amazon_reviews
.. automodule:: recommenders.datasets.amazon_reviews
:members:

CORD-19
Expand All @@ -44,7 +44,7 @@ to generate new insights in support of the fight against this infectious disease
Funk, K., Kinney, R., Liu, Z., Merrill, W. and Mooney, P. "Cord-19: The COVID-19 Open Research Dataset.", 2020.


.. automodule:: reco_utils.datasets.covid_utils
.. automodule:: recommenders.datasets.covid_utils
:members:

Criteo
Expand All @@ -55,7 +55,7 @@ for millions of display Ads. Every Ad has has 40 attributes, the first attribute
that the Ad has been clicked on and a 0 represents it wasn't clicked on. The rest consist of 13 integer columns and
26 categorical columns.

.. automodule:: reco_utils.datasets.criteo
.. automodule:: recommenders.datasets.criteo
:members:

MIND
Expand All @@ -76,7 +76,7 @@ this impression. To protect user privacy, each user was de-linked from the produ



.. automodule:: reco_utils.datasets.mind
.. automodule:: recommenders.datasets.mind
:members:

MovieLens
Expand All @@ -101,52 +101,52 @@ It comes with several sizes:
ACM Transactions on Interactive Intelligent Systems (TiiS) 5, 4, Article 19,
DOI=http://dx.doi.org/10.1145/2827872, 2015.

.. automodule:: reco_utils.datasets.movielens
.. automodule:: recommenders.datasets.movielens
:members:

Download utilities
******************

.. automodule:: reco_utils.datasets.download_utils
.. automodule:: recommenders.datasets.download_utils
:members:


Cosmos CLI utilities
*********************

.. automodule:: reco_utils.datasets.cosmos_cli
.. automodule:: recommenders.datasets.cosmos_cli
:members:


Pandas dataframe utilities
***************************

.. automodule:: reco_utils.datasets.pandas_df_utils
.. automodule:: recommenders.datasets.pandas_df_utils
:members:


Splitter utilities
******************

.. automodule:: reco_utils.datasets.python_splitters
.. automodule:: recommenders.datasets.python_splitters
:members:

.. automodule:: reco_utils.datasets.spark_splitters
.. automodule:: recommenders.datasets.spark_splitters
:members:

.. automodule:: reco_utils.datasets.split_utils
.. automodule:: recommenders.datasets.split_utils
:members:


Sparse utilities
****************

.. automodule:: reco_utils.datasets.sparse
.. automodule:: recommenders.datasets.sparse
:members:


Knowledge graph utilities
*************************

.. automodule:: reco_utils.datasets.wikidata
.. automodule:: recommenders.datasets.wikidata
:members:
4 changes: 2 additions & 2 deletions docs/source/evaluation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@ Evaluation module
Python evaluation
===============================

.. automodule:: reco_utils.evaluation.python_evaluation
.. automodule:: recommenders.evaluation.python_evaluation
:members:

PySpark evaluation
===============================

.. automodule:: reco_utils.evaluation.spark_evaluation
.. automodule:: recommenders.evaluation.spark_evaluation
:members:
2 changes: 1 addition & 1 deletion docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Recommender Utilities

The `Recommenders repository <https://github.com/Microsoft/Recommenders>`_ provides examples and best practices for building recommendation systems, provided as Jupyter notebooks.

The module `reco_utils <https://github.com/microsoft/recommenders/tree/main/reco_utils>`_ contains functions to simplify common tasks used when developing and
The module `recommenders <https://github.com/microsoft/recommenders/tree/main/recommenders>`_ contains functions to simplify common tasks used when developing and
evaluating recommender systems.

.. toctree::
Expand Down
Loading