Skip to content

This is a prototype version of the Kedro spaceflights tutorial that utilises the modular pipelines pattern.

Notifications You must be signed in to change notification settings

datajoely/modular-spaceflights

Repository files navigation

modular-spaceflights

This project is designed to be a realistic example of what Kedro looks like when used in anger.

Setup

  1. Run pip install kedro==0.17.6
  2. Run kedro install --build-reqs
  3. Run kedro run
  4. Run kedro viz

Visualised output

Visualised output

Touring through the codebase

This example project has been built to demonstrate several key journeys. Each of these has an associated VS Code Tour that takes you around the interface to learn how it has been structured and to demonstrate some of the advanced parts of Kedro in way that reflects real world usage.

# Task Description
1 Building the ingestion pipeline 🏗 Takes you through a simple ingestion pipeline which types the data and utilises key features such as parameters and namespaces.
2 Engineering features 🧩 Simple feature engineering pipeline which utilises some advanced techniques for deriving new features.
3 Running the models 🧠 A modelling pipeline that creates two instances of train/evaluation pipelines parametrised each applying a different modelling technique.
4 Plotly plots in Kedro Viz 📈 A walk-through on how to render Plotly visualisations natively in Kedro-Viz.
5 Custom DataSets 💾 This is a short tutorial on how to define and utilise a custom dataset by inheriting from Kedro's AbstractDataSet type.
6 Custom Hooks 🎣 Hooks allow you to add custom functionality to the kedro run lifecycle. This tutorial takes you through an example hook which records the time taken to read data from disk and logs out the durations.
7 Additional Configuration Environments ⚙️ One of the most powerful features in Kedro is the way that one can introduce hierarchical configurations. This follows the philosophy set out in the 12FactorApp.

How to use the VS Code Tour Extension

extension

Method Description
Install VS Code + Extension For best results install the application and extension locally.
Use the gitHub.dev web based editor For quickest results simply press . in the GitHub repository and install the Code Tour extension from the marketplace using the web-based editor. At the time of writing embedded images don't work, but everything else does.

About

This is a prototype version of the Kedro spaceflights tutorial that utilises the modular pipelines pattern.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published