A file router for distributed systems, which (just) handles routing logic. Like a railway switch, this tool (neither) pulls nor pushes files (yet), it simply evaluates their names, meta-data or content and provides a route to get the files to their destination.
While the core idea - providing a simple, testable and extendable logic for file routings - of this project has not changed, features to actually move files will be added from now on.
This extends the projects uses cases to the underlying problem: Connecting different file sources and destinations. By implementing both, we allow ETL-tools like Prefect and Airflow to utilize their proficiency in orchestrating and executing task/flows, while centralizing or separating the underlying routing logic and services with this package.
A standalone application, overseeing the complete routing process - from source to destination, is currently out of scope but may be implemented in the future. PR are more than welcome!
As usual, use a virtualenv, and install via pip or pipenv. Because this package is not yet published on PyPi, please install the latest stable version from GitHub or from source.
pip install git+https://github.com/mj0nez/FileSwitch.gitAlternatively, clone this project and install in development mode:
git clone https://github.com/mj0nez/FileSwitch.git
cd FileSwitch
pip install -e .[dev]Follow the above mentioned dev-installation. Format all files with black.
PR are more than welcome to make this library better, or to add a feature that matches your needs. Nevertheless, don't forget adding tests for every aspect you. This project uses pytest.
This library is licensed under the MIT license, see the LICENSE file.