Update docs

docs/api_reference/index.md

@@ -1,33 +1,9 @@
-# Python API Reference
-
-Welcome to the API reference for the Python/Cython implementation of NautilusTrader!
-
-The API reference provides detailed technical documentation for the NautilusTrader framework,
-including its modules, classes, methods, and functions. The reference is automatically generated
-from the latest NautilusTrader source code using [Sphinx](https://www.sphinx-doc.org/en/master/).
-
-Please note that there are separate references for different versions of NautilusTrader:
-
-- **Latest**: This reference is built from the head of the `master` branch and represents the documentation for the latest stable release.
-- **Develop**: This reference is built from the head of the `develop` branch and represents the documentation for the latest changes and features currently in development.
-
-You can select the desired API reference from the **Versions** top right drop down menu.
-
-```{note}
-If you select an item from the top level navigation, this will take you to the **Latest** API reference.
-```
-
-Use the right navigation sidebar to explore the available modules and their contents.
-You can click on any item to view its detailed documentation, including parameter descriptions, and return value explanations.
-
-If you have any questions or need further assistance, please reach out to the NautilusTrader community for support.
+# Python API
 
 ```{eval-rst}
 .. toctree::
    :maxdepth: 1
    :glob:
-   :titlesonly:
-   :hidden:
    
    accounting.md
    adapters/index.md
@@ -51,3 +27,36 @@ If you have any questions or need further assistance, please reach out to the Na
    system.md
    trading.md
 ```
+
+Welcome to the Python API reference for NautilusTrader!
+
+The API reference provides detailed technical documentation for the NautilusTrader framework,
+including its modules, classes, methods, and functions. The reference is automatically generated
+from the latest NautilusTrader source code using [Sphinx](https://www.sphinx-doc.org/en/master/).
+
+Please note that there are separate references for different versions of NautilusTrader:
+
+- **Latest**: This API reference is built from the head of the `master` branch and represents the latest stable release.
+- **Develop**: This API reference is built from the head of the `develop` branch and represents bleeding edge and experimental changes/features currently in development.
+
+You can select the desired API reference from the **Versions** top right drop down menu.
+
+```{note}
+If you select an item from the top level navigation, this will take you to the **Latest** API reference.
+```
+
+Use the right navigation sidebar to explore the available modules and their contents.
+You can click on any item to view its detailed documentation, including parameter descriptions, and return value explanations.
+
+## Why Python?
+
+Python was originally created decades ago as a simple scripting language with a clean straight
+forward syntax. It has since evolved into a fully fledged general purpose object-oriented
+programming language. Based on the TIOBE index, Python is currently the most popular programming language in the world.
+Not only that, Python has become the _de facto lingua franca_ of data science, machine learning, and artificial intelligence.
+
+The language out of the box is not without its drawbacks however, especially in the context of
+implementing large performance-critical systems. Cython has addressed a lot of these issues, offering all the advantages
+of a statically typed language, embedded into Pythons rich ecosystem of software libraries and
+developer/user communities.
+

docs/concepts/architecture.md

@@ -45,6 +45,8 @@ Throughout the documentation, the term _"Nautilus system boundary"_ refers to op
 the runtime of a single Nautilus node (also known as a "trader instance").
 ```
 
+![Architecture](https://github.com/nautechsystems/nautilus_trader/blob/develop/docs/_images/architecture-overview.png?raw=true "architecture")
+
 ### Environment contexts
 - `Backtest` - Historical data with simulated venues
 - `Sandbox` - Real-time data with simulated venues

docs/concepts/data.md

@@ -45,4 +45,11 @@ Conceretely, this would involve for example:
 
 ## Data catalog
 
+The data catalog is a central store for Nautilus data, persisted in the [Parquet](https://parquet.apache.org) file format.
+
+We have chosen parquet as the storage format for the following reasons:
+- It performs much better than CSV/JSON/HDF5/etc in terms of compression ratio (storage size) and read performance
+- It does not require any separate running components (for example a database)
+- It is quick and simple to get up and running with
+
 **This doc is an evolving work in progress and will continue to describe the data catalog more fully...**

docs/concepts/index.md

@@ -26,12 +26,6 @@ Welcome to NautilusTrader!
 Explore the foundational concepts of NautilusTrader through the following guides.
 
 ```{note}
-It's important to note that the [API Reference](../api_reference/index.md) documentation should be 
-considered the source of truth for the platform. If there are any discrepancies between concepts described here
-and the API Reference, then the API Reference should be considered the correct information. We are 
-working to ensure that concepts stay up-to-date with the API Reference and will be introducing 
-doc tests in the near future to help with this.
-
 The terms "NautilusTrader", "Nautilus" and "platform" are used interchageably throughout the documentation.
 ```
 
@@ -77,3 +71,11 @@ The platform provides logging for both backtesting and live trading using a high
 ## [Advanced](advanced/index.md)
 Here you will find more detailed documentation and examples covering the more advanced
 features and functionality of the platform.
+
+```{note}
+It's important to note that the [API Reference](../api_reference/index.md) documentation should be 
+considered the source of truth for the platform. If there are any discrepancies between concepts described here
+and the API Reference, then the API Reference should be considered the correct information. We are 
+working to ensure that concepts stay up-to-date with the API Reference and will be introducing 
+doc tests in the near future to help with this.
+```

docs/concepts/overview.md

@@ -1,5 +1,58 @@
 # Overview
 
+NautilusTrader is an open-source, high-performance, production-grade algorithmic trading platform,
+providing quantitative traders with the ability to backtest portfolios of automated trading strategies
+on historical data with an event-driven engine, and also deploy those same strategies live, with no code changes.
+
+The platform is 'AI-first', designed to develop and deploy algorithmic trading strategies within a highly performant 
+and robust Python native environment. This helps to address the parity challenge of keeping the Python research/backtest 
+environment, consistent with the production live trading environment.
+
+NautilusTraders design, architecture and implementation philosophy holds software correctness and safety at the
+highest level, with the aim of supporting Python native, mission-critical, trading system backtesting
+and live deployment workloads.
+
+The platform is also universal and asset class agnostic - with any REST, WebSocket or FIX API able to be integrated via modular
+adapters. Thus, it can handle high-frequency trading operations for any asset classes
+including FX, Equities, Futures, Options, CFDs, Crypto and Betting - across multiple venues simultaneously.
+
+## Features
+
+- **Fast:** C-level speed through Rust and Cython. Asynchronous networking with [uvloop](https://github.com/MagicStack/uvloop)
+- **Reliable:** Type safety through Rust and Cython. Redis backed performant state persistence
+- **Flexible:** OS independent, runs on Linux, macOS, Windows. Deploy using Docker
+- **Integrated:** Modular adapters mean any REST, WebSocket, or FIX API can be integrated
+- **Advanced:** Time in force `IOC`, `FOK`, `GTD`, `AT_THE_OPEN`, `AT_THE_CLOSE`, advanced order types and conditional triggers. Execution instructions `post-only`, `reduce-only`, and icebergs. Contingency order lists including `OCO`, `OTO`
+- **Backtesting:** Run with multiple venues, instruments and strategies simultaneously using historical quote tick, trade tick, bar, order book and custom data with nanosecond resolution
+- **Live:** Use identical strategy implementations between backtesting and live deployments
+- **Multi-venue:** Multiple venue capabilities facilitate market making and statistical arbitrage strategies
+- **AI Agent Training:** Backtest engine fast enough to be used to train AI trading agents (RL/ES)
+
+![Nautilus](https://github.com/nautechsystems/nautilus_trader/blob/develop/docs/_images/nautilus-art.png?raw=true "nautilus")
+> *nautilus - from ancient Greek 'sailor' and naus 'ship'.*
+>
+> *The nautilus shell consists of modular chambers with a growth factor which approximates a logarithmic spiral.
+> The idea is that this can be translated to the aesthetics of design and architecture.*
+
+## Why NautilusTrader?
+
+- **Highly performant event-driven Python** - native binary core components
+- **Parity between backtesting and live trading** - identical strategy code
+- **Reduced operational risk** - risk management functionality, logical correctness and type safety
+- **Highly extendable** - message bus, custom components and actors, custom data, custom adapters
+
+Traditionally, trading strategy research and backtesting might be conducted in Python (or other suitable language)
+using vectorized methods, with the strategy then needing to be reimplemented in a more event-drive way
+using C++, C#, Java or other statically typed language(s). The reasoning here is that vectorized backtesting code cannot
+express the granular time and event dependent complexity of real-time trading, where compiled languages have
+proven to be more suitable due to their inherently higher performance, and type safety.
+
+One of the key advantages of NautilusTrader here, is that this reimplementation step is now circumvented - as the critical core components of the platform
+have all been written entirely in Rust or Cython. This means we're using the right tools for the job, where systems programming languages compile performant binaries, 
+with CPython C extension modules then able to offer a Python native environment, suitable for professional quantitative traders and trading firms.
+
+## Use cases
+
 There are three main use cases for this software package:
 
 - Backtesting trading systems with historical data (`backtest`)

docs/conf.py

@@ -80,17 +80,17 @@
             "title": "Getting Started",
         },
         {
-            "href": "/user_guide/index",
+            "href": "/concepts/index",
             "internal": True,
-            "title": "User Guide",
+            "title": "Concepts",
         },
         {
             "href": "/api_reference/index",
             "internal": True,
             "title": "Python API",
         },
         {
-            "href": "/core/index",
+            "href": "rust",
             "internal": True,
             "title": "Rust API",
         },

docs/developer_guide/cython.md

@@ -3,6 +3,16 @@
 Here you will find guidance and tips for working on NautilusTrader using the Cython language.
 More information on Cython syntax and conventions can be found by reading the [Cython docs](https://cython.readthedocs.io/en/latest/index.html).
 
+## What is Cython?
+
+[Cython](https://cython.org) is a compiled programming language that aims to be a superset of the Python programming
+language, designed to give C-like performance with code that is written mostly in Python with
+optional additional C-inspired syntax.
+
+The project heavily utilizes Cython to provide static type safety and increased performance
+for Python through [C extension modules](https://docs.python.org/3/extending/extending.html). The vast majority of the production code is actually
+written in Cython, however the libraries can be accessed from both Python and Cython.
+
 ## Function and method signatures
 Ensure that all functions and methods returning `void` or a primitive C type (such as `bint`, `int`, `double`) include the `except *` keyword in the signature.
 

docs/developer_guide/index.md

@@ -1,5 +1,18 @@
 # Developer Guide
 
+```{eval-rst}
+.. toctree::
+   :maxdepth: 2
+   :hidden:
+   
+   environment_setup.md
+   coding_standards.md
+   cython.md
+   rust.md
+   testing.md
+   packaged_data.md
+```
+
 Welcome to the developer guide for NautilusTrader!
 
 Here you will find information related to developing and extending the NautilusTrader codebase. 
@@ -31,19 +44,6 @@ It's not necessary to become a C language expert, however it's helpful to unders
 syntax is used in function and method definitions, in local code blocks, and the common primitive C 
 types and how these map to their corresponding `PyObject` types.
 
-```{eval-rst}
-.. toctree::
-   :maxdepth: 2
-   :hidden:
-   
-   environment_setup.md
-   coding_standards.md
-   cython.md
-   rust.md
-   testing.md
-   packaged_data.md
-```
-
 ## Contents
 
 - [Environment Setup](environment_setup.md)

docs/getting_started/index.md

@@ -7,16 +7,13 @@
    :titlesonly:
    :hidden:
    
-   introduction.md
    installation.md
    quickstart.md
 ```
 
-Welcome to the NautilusTrader getting started section!
-
-## [Introduction](introduction.md)
-The **Introduction** covers the value proposition for the platform and why NautilusTrader exists, as 
-well as a very high-level summary of the main features.
+To get started with NautilusTrader you will need the following:
+- A Python environment with `nautilus_trader` installed
+- A way to launch Python scripts for backtesting and live trading (either from the command line, or jupyter notebook etc)
 
 ## [Installation](installation.md)
 The **Installation** guide will help to ensure that NautilusTrader is properly installed on your machine.