Release 1.188.0

.gitignore

@@ -18,6 +18,9 @@
 *.tar.gz*
 *.zip
 
+*.dbn
+*.dbn.zst
+
 .benchmarks*
 .coverage*
 .history*
@@ -34,6 +37,7 @@
 .vscode/
 
 /catalog/
+/examples/notebooks/catalog/
 
 __pycache__
 _build/

.pre-commit-config.yaml

@@ -73,7 +73,7 @@ repos:
         types: [python]
 
   - repo: https://github.com/psf/black
-    rev: 24.1.1
+    rev: 24.2.0
     hooks:
       - id: black
         types_or: [python, pyi]
@@ -82,7 +82,7 @@ repos:
         exclude: "docs/_pygments/monokai.py"
 
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.2.1
+    rev: v0.2.2
     hooks:
       - id: ruff
         args: ["--fix"]

RELEASES.md

@@ -1,3 +1,47 @@
+# NautilusTrader 1.188.0 Beta
+
+Released on 25th February 2024 (UTC).
+
+### Enhancements
+- Added `FuturesSpread` instrument type
+- Added `OptionsSpread` instrument type
+- Added `InstrumentClass.FUTURE_SPREAD`
+- Added `InstrumentClass.OPTION_SPREAD`
+- Added `managed` parameter to `subscribe_order_book_deltas`, default true to retain current behavior (if false then the data engine will not automatically manage a book)
+- Added `managed` parameter to `subscribe_order_book_snapshots`, default true to retain current behavior (if false then the data engine will not automatically manage a book)
+- Added additional validations for `OrderMatchingEngine` (will now reject orders with incorrect price or quantity precisions)
+- Removed `interval_ms` 20 millisecond limitation for `subscribe_order_book_snapshots` (i.e. just needs to be positive), although we recommend you consider subscribing to deltas below 100 milliseconds
+- Ported `LiveClock` and `LiveTimer` implementations to Rust
+- Implemented `OrderBookDeltas` pickling
+- Implemented `AverageTrueRange` in Rust, thanks @rsmb7z
+
+### Breaking Changes
+- Changed `TradeId` value maximum length to 36 characters (will raise a `ValueError` if value exceeds the maximum)
+
+### Fixes
+- Fixed `TradeId` memory leak due assigning unique values to the `Ustr` global string cache (which are never freed for the lifetime of the program)
+- Fixed `TradeTick` size precision for pyo3 conversion (size precision was incorrectly price precision)
+- Fixed `RiskEngine` cash value check when selling (would previously divide quantity by price which is too much), thanks for reporting@AnthonyVince
+- Fixed FOK time in force behavior (allows fills beyond the top level, will cancel if cannot fill full size)
+- Fixed IOC time in force behavior (allows fills beyond the top level, will cancel any remaining after all fills are applied)
+- Fixed `LiveClock` timer behavior for small intervals causing next time to be less than now (timer then would not run)
+- Fixed log level filtering for `log_level_file` (bug introduced in v1.187.0), thanks @twitu
+- Fixed logging `print_config` config option (was not being passed through to the logging system)
+- Fixed logging timestamps for backtesting (static clock was not being incrementally set to individual `TimeEvent` timestamps)
+- Fixed account balance updates (fills from zero quantity `NETTING` positions will generate account balance updates)
+- Fixed `MessageBus` publishable types collection type (needed to be `tuple` not `set`)
+- Fixed `Controller` registration of components to ensure all active clocks are iterated correctly during backtests
+- Fixed `Equity` short selling for `CASH` accounts (will now reject)
+- Fixed `ActorFactory.create` JSON encoding (was missing the encoding hook)
+- Fixed `ImportableConfig.create` JSON encoding (was missing the encoding hook)
+- Fixed `ImportableStrategyConfig.create` JSON encoding (was missing the encoding hook)
+- Fixed `ExecAlgorithmFactory.create` JSON encoding (was missing the encoding hook)
+- Fixed `ControllerConfig` base class and docstring
+- Fixed Interactive Brokers historical bar data bug, thanks @benjaminsingleton
+- Fixed persistence `freeze_dict` function to handle `fs_storage_options`, thanks @dimitar-petrov
+
+---
+
 # NautilusTrader 1.187.0 Beta
 
 Released on 9th February 2024 (UTC).

examples/live/binance/binance_spot_orderbook_imbalance_rust.py

@@ -0,0 +1,119 @@
+#!/usr/bin/env python3
+# -------------------------------------------------------------------------------------------------
+#  Copyright (C) 2015-2024 Nautech Systems Pty Ltd. All rights reserved.
+#  https://nautechsystems.io
+#
+#  Licensed under the GNU Lesser General Public License Version 3.0 (the "License");
+#  You may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at https://www.gnu.org/licenses/lgpl-3.0.en.html
+#
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+# -------------------------------------------------------------------------------------------------
+
+from decimal import Decimal
+
+from nautilus_trader.adapters.binance.common.enums import BinanceAccountType
+from nautilus_trader.adapters.binance.config import BinanceDataClientConfig
+from nautilus_trader.adapters.binance.config import BinanceExecClientConfig
+from nautilus_trader.adapters.binance.factories import BinanceLiveDataClientFactory
+from nautilus_trader.adapters.binance.factories import BinanceLiveExecClientFactory
+from nautilus_trader.config import CacheConfig
+from nautilus_trader.config import InstrumentProviderConfig
+from nautilus_trader.config import LiveExecEngineConfig
+from nautilus_trader.config import LoggingConfig
+from nautilus_trader.config import TradingNodeConfig
+from nautilus_trader.examples.strategies.orderbook_imbalance_rust import OrderBookImbalance
+from nautilus_trader.examples.strategies.orderbook_imbalance_rust import OrderBookImbalanceConfig
+from nautilus_trader.live.node import TradingNode
+from nautilus_trader.model.identifiers import InstrumentId
+from nautilus_trader.model.identifiers import TraderId
+
+
+# *** THIS IS A TEST STRATEGY WITH NO ALPHA ADVANTAGE WHATSOEVER. ***
+# *** IT IS NOT INTENDED TO BE USED TO TRADE LIVE WITH REAL MONEY. ***
+
+
+# Configure the trading node
+config_node = TradingNodeConfig(
+    trader_id=TraderId("TESTER-001"),
+    logging=LoggingConfig(
+        log_level="INFO",
+        # log_level_file="DEBUG",
+        # log_file_format="json",
+    ),
+    exec_engine=LiveExecEngineConfig(
+        reconciliation=True,
+        reconciliation_lookback_mins=1440,
+        filter_position_reports=True,
+    ),
+    cache=CacheConfig(
+        database=None,
+        timestamps_as_iso8601=True,
+        flush_on_start=False,
+    ),
+    # snapshot_orders=True,
+    # snapshot_positions=True,
+    # snapshot_positions_interval=5.0,
+    data_clients={
+        "BINANCE": BinanceDataClientConfig(
+            api_key=None,  # 'BINANCE_API_KEY' env var
+            api_secret=None,  # 'BINANCE_API_SECRET' env var
+            account_type=BinanceAccountType.SPOT,
+            base_url_http=None,  # Override with custom endpoint
+            base_url_ws=None,  # Override with custom endpoint
+            us=False,  # If client is for Binance US
+            testnet=False,  # If client uses the testnet
+            instrument_provider=InstrumentProviderConfig(load_all=True),
+        ),
+    },
+    exec_clients={
+        "BINANCE": BinanceExecClientConfig(
+            api_key=None,  # 'BINANCE_API_KEY' env var
+            api_secret=None,  # 'BINANCE_API_SECRET' env var
+            account_type=BinanceAccountType.SPOT,
+            base_url_http=None,  # Override with custom endpoint
+            base_url_ws=None,  # Override with custom endpoint
+            us=False,  # If client is for Binance US
+            testnet=False,  # If client uses the testnet
+            instrument_provider=InstrumentProviderConfig(load_all=True),
+        ),
+    },
+    timeout_connection=20.0,
+    timeout_reconciliation=10.0,
+    timeout_portfolio=10.0,
+    timeout_disconnection=10.0,
+    timeout_post_stop=5.0,
+)
+
+# Instantiate the node with a configuration
+node = TradingNode(config=config_node)
+
+# Configure your strategy
+strat_config = OrderBookImbalanceConfig(
+    instrument_id=InstrumentId.from_str("ETHUSDT.BINANCE"),
+    external_order_claims=[InstrumentId.from_str("ETHUSDT.BINANCE")],
+    max_trade_size=Decimal("0.010"),
+)
+
+# Instantiate your strategy
+strategy = OrderBookImbalance(config=strat_config)
+
+# Add your strategies and modules
+node.trader.add_strategy(strategy)
+
+# Register your client factories with the node (can take user defined factories)
+node.add_data_client_factory("BINANCE", BinanceLiveDataClientFactory)
+node.add_exec_client_factory("BINANCE", BinanceLiveExecClientFactory)
+node.build()
+
+
+# Stop and dispose of the node with SIGINT/CTRL+C
+if __name__ == "__main__":
+    try:
+        node.run()
+    finally:
+        node.dispose()

examples/live/databento/databento_subscriber.py

@@ -14,10 +14,12 @@
 #  limitations under the License.
 # -------------------------------------------------------------------------------------------------
 
-from nautilus_trader.adapters.databento.config import DatabentoDataClientConfig
-from nautilus_trader.adapters.databento.constants import DATABENTO
-from nautilus_trader.adapters.databento.constants import DATABENTO_CLIENT_ID
-from nautilus_trader.adapters.databento.factories import DatabentoLiveDataClientFactory
+from nautilus_trader.adapters.databento import DATABENTO
+from nautilus_trader.adapters.databento import DATABENTO_CLIENT_ID
+from nautilus_trader.adapters.databento import DatabentoDataClientConfig
+from nautilus_trader.adapters.databento import DatabentoLiveDataClientFactory
+from nautilus_trader.cache.config import CacheConfig
+from nautilus_trader.common.config import DatabaseConfig
 from nautilus_trader.common.enums import LogColor
 from nautilus_trader.config import InstrumentProviderConfig
 from nautilus_trader.config import LiveExecEngineConfig
@@ -43,8 +45,9 @@
 # For correct subscription operation, you must specify all instruments to be immediately
 # subscribed for as part of the data client configuration
 instrument_ids = [
-    InstrumentId.from_str("ESH4.GLBX"),
-    # InstrumentId.from_str("ESM4.GLBX"),
+    InstrumentId.from_str("ESH4.XCME"),
+    # InstrumentId.from_str("ESM4.XCME"),
+    # InstrumentId.from_str("ESU4.XCME"),
     # InstrumentId.from_str("AAPL.XCHI"),
 ]
 
@@ -56,12 +59,12 @@
         reconciliation=False,  # Not applicable
         inflight_check_interval_ms=0,  # Not applicable
     ),
-    # cache=CacheConfig(
-    #     database=DatabaseConfig(),
-    #     encoding="json",
-    #     timestamps_as_iso8601=True,
-    #     buffer_interval_ms=100,
-    # ),
+    cache=CacheConfig(
+        database=DatabaseConfig(),
+        encoding="msgpack",
+        timestamps_as_iso8601=True,
+        buffer_interval_ms=100,
+    ),
     # message_bus=MessageBusConfig(
     #     database=DatabaseConfig(),
     #     encoding="json",

examples/notebooks/backtest_binance_orderbook.ipynb

@@ -7,7 +7,7 @@
    "source": [
     "# Backtest on Binance OrderBook data\n",
     "\n",
-    "This example runs through how to setup the data catalog and a `BacktestNode` to backtest an `OrderBookImbalance` strategy or order book data. This example requires you bring your Binance own order book data.\n",
+    "This tutorial runs through how to setup the data catalog and a `BacktestNode` to backtest an `OrderBookImbalance` strategy or order book data. This example requires you bring your Binance own order book data.\n",
     "\n",
     "**Warning:**\n",
     "\n",
@@ -41,14 +41,17 @@
     "import pandas as pd\n",
     "\n",
     "from nautilus_trader.backtest.node import BacktestNode\n",
+    "from nautilus_trader.backtest.node import BacktestVenueConfig\n",
+    "from nautilus_trader.backtest.node import BacktestDataConfig\n",
+    "from nautilus_trader.backtest.node import BacktestRunConfig\n",
+    "from nautilus_trader.backtest.node import BacktestEngineConfig\n",
     "from nautilus_trader.core.datetime import dt_to_unix_nanos\n",
-    "from nautilus_trader.config import BacktestRunConfig, BacktestVenueConfig, BacktestDataConfig, BacktestEngineConfig\n",
     "from nautilus_trader.config import ImportableStrategyConfig\n",
     "from nautilus_trader.config import LoggingConfig\n",
     "from nautilus_trader.examples.strategies.ema_cross import EMACross, EMACrossConfig\n",
     "from nautilus_trader.model.data import OrderBookDelta\n",
     "from nautilus_trader.persistence.loaders import BinanceOrderBookDeltaDataLoader\n",
-    "from nautilus_trader.persistence.wranglers import OrderBookDeltaDataWranglerV2\n",
+    "from nautilus_trader.persistence.wranglers import OrderBookDeltaDataWrangler\n",
     "from nautilus_trader.persistence.catalog import ParquetDataCatalog\n",
     "from nautilus_trader.test_kit.providers import TestInstrumentProvider"
    ]
@@ -113,7 +116,7 @@
    "source": [
     "# Process deltas using a wrangler\n",
     "BTCUSDT_BINANCE = TestInstrumentProvider.btcusdt_binance()\n",
-    "wrangler = OrderBookDeltaDataWranglerV2(BTCUSDT_BINANCE)\n",
+    "wrangler = OrderBookDeltaDataWrangler(BTCUSDT_BINANCE)\n",
     "\n",
     "deltas = wrangler.process(df_snap)\n",
     "deltas += wrangler.process(df_update)\n",
@@ -146,7 +149,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# Write instrument and ticks to catalog (this currently takes a minute - investigating)\n",
+    "# Write instrument and ticks to catalog\n",
     "catalog.write_data([BTCUSDT_BINANCE])\n",
     "catalog.write_data(deltas)"
    ]

examples/notebooks/backtest_example.ipynb

@@ -5,7 +5,7 @@
    "id": "0",
    "metadata": {},
    "source": [
-    "# Complete backtest using the data catalog and a BacktestNode (higher level)\n",
+    "# Complete backtest using the data catalog and a BacktestNode (high-level API)\n",
     "\n",
     "This example runs through how to setup the data catalog and a `BacktestNode` for a single 'one-shot' backtest run."
    ]
@@ -32,8 +32,11 @@
     "import pandas as pd\n",
     "\n",
     "from nautilus_trader.backtest.node import BacktestNode\n",
+    "from nautilus_trader.backtest.node import BacktestVenueConfig\n",
+    "from nautilus_trader.backtest.node import BacktestDataConfig\n",
+    "from nautilus_trader.backtest.node import BacktestRunConfig\n",
+    "from nautilus_trader.backtest.node import BacktestEngineConfig\n",
     "from nautilus_trader.core.datetime import dt_to_unix_nanos\n",
-    "from nautilus_trader.config import BacktestRunConfig, BacktestVenueConfig, BacktestDataConfig, BacktestEngineConfig\n",
     "from nautilus_trader.config import ImportableStrategyConfig\n",
     "from nautilus_trader.config import LoggingConfig\n",
     "from nautilus_trader.examples.strategies.ema_cross import EMACross, EMACrossConfig\n",
@@ -170,6 +173,14 @@
    "source": [
     "result"
    ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "10",
+   "metadata": {},
+   "outputs": [],
+   "source": []
   }
  ],
  "metadata": {

examples/notebooks/backtest_fx_usdjpy.ipynb

@@ -5,9 +5,9 @@
    "id": "0",
    "metadata": {},
    "source": [
-    "# Complete backtest using a wrangler and BacktestEngine (lower level)\n",
+    "# Complete backtest using a wrangler and BacktestEngine (low-level API)\n",
     "\n",
-    "This example runs through how to setup a `BacktestEngine` for a single 'one-shot' backtest run."
+    "This tutorial runs through how to setup a `BacktestEngine` for a single 'one-shot' backtest run."
    ]
   },
   {