ADR 18: Keeping a single state

docs/adr/2022-03-23_018-single-state.md

@@ -0,0 +1,85 @@
+---
+slug: 18
+title: | 
+  18. Single state in Hydra.Node.
+authors: []
+tags: [Proposed]
+---
+
+## Status
+
+Proposed
+
+## Context
+
+* Currently the `hydra-node` maintains two pieces of state during the life-cycle of a Hydra Head:
+  1. A `HeadState tx` provided by the `HydraHead tx m` handle interface and part of the `Hydra.Node` module. It provides the basis for the main `hydra-node` business logic in `Hydra.Node.processNextEvent` and `Hydra.HeadLogic.update`
+  [Creation](https://github.com/input-output-hk/hydra-poc/blob/a98e2907c4e425de2736782793383aad63132c14/hydra-node/src/Hydra/Node.hs#L256-L257), [Usage](https://github.com/input-output-hk/hydra-poc/blob/a98e2907c4e425de2736782793383aad63132c14/hydra-node/src/Hydra/Node.hs#L174)
+  2. `SomeOnChainHeadState` is kept in the `Hydra.Chain.Direct` to keep track of the latest known head state, including notable transaction outputs and information how to spend it (e.g. scripts and datums)
+  [Code](https://github.com/input-output-hk/hydra-poc/blob/a98e2907c4e425de2736782793383aad63132c14/hydra-node/src/Hydra/Chain/Direct.hs#L156-L162), [Usage 1](https://github.com/input-output-hk/hydra-poc/blob/a98e2907c4e425de2736782793383aad63132c14/hydra-node/src/Hydra/Chain/Direct.hs#L449), [Usage 2](https://github.com/input-output-hk/hydra-poc/blob/a98e2907c4e425de2736782793383aad63132c14/hydra-node/src/Hydra/Chain/Direct.hs#L414), [Usage 3](https://github.com/input-output-hk/hydra-poc/blob/a98e2907c4e425de2736782793383aad63132c14/hydra-node/src/Hydra/Chain/Direct.hs#L349-L352)
+  (There are other unrelated things kept in memory like the event history in the API server or a peer map in the network heartbeat component.)
+* The interface between the `Hydra.Node` and a `Hydra.Chain` component consists of 
+  - constructing certain Head protocol transactions given a description of it (`PostChainTx tx`):
+    ```hs
+    postTx :: MonadThrow m => PostChainTx tx -> m ()
+    ```
+  - a callback function when the `Hydra.Chain` component observed a new Head protocol transaction described by `OnChainTx tx`:
+    ```hs
+    type ChainCallback tx m = OnChainTx tx -> m ()
+    ```
+* Given by the usage sites above, the `Hydra.Chain.Direct` module requires additional info to do both, construct protocol transactions with `postTx` as well as observe potential `OnChainTx` ([here](https://github.com/input-output-hk/hydra-poc/blob/a98e2907c4e425de2736782793383aad63132c14/hydra-node/src/Hydra/Chain/Direct.hs#L333-L336)). Hence we see that, operation of the `Hydra.Chain.Direct` component (and likely any implementing the interface fully) is **inherently stateful**.
+* We are looking at upcoming features to [handle rollbacks](https://github.com/input-output-hk/hydra-poc/issues/185) and dealing with [persisting the head state](https://github.com/input-output-hk/hydra-poc/issues/187).
+  - Both could benefit from the idea, that the `HeadState` is just a result of pure `Event` processing (a.k.a event sourcing).
+  - Right now the `HeadState` kept in `Hydra.Node` alone, is not enough to fully describe the state of the `hydra-node`. Hence it would not be enough to just persist all the `Event`s and replaying them to achieve persistence, nor resetting to some previous `HeadState` in the presence of a rollback.
+
+## Decision
+
+* We define and keep a "blackbox" `ChainState tx` in the `HeadState tx`
+  - It shall not be introspectable to the business logic in `HeadLogic`
+  - It shall contain chain-specific information about the current Hydra Head, which will naturally need to evolve once we have multiple Heads in our feature scope
+  - For example:
+  ```hs
+  data HeadState tx
+    = ReadyState
+    | InitialState
+        { chainState :: ChainState tx
+        -- ...
+        }
+    | OpenState
+        { chainState :: ChainState tx
+        -- ...
+        }
+    | ClosedState
+        { chainState :: ChainState tx
+        -- ...
+        }
+  ```
+* We provide the latest `ChainState tx` to `postTx`:
+  ```hs
+  postTx :: ChainState tx -> PostChainTx tx -> m ()
+  ```
+* We change the callback interface of `Chain` to
+  ```hs
+  type ChainCallback tx m = (ChainState tx -> Maybe (OnChainTx tx, ChainState tx)) -> m ()
+  ```
+with the meaning, that invoking the callback indicates receival of a potential Hydra transaction which is `Maybe` observing a relevant `OnChainTx tx` paired with a (potentially updated) `ChainState tx`.
+* We also decide to extend `OnChainEvent` and `OnChainEffect` with a `ChainState tx` and threading it through the `Hydra.HeadLogic`.
+
+## Consequences
+
+* We need to change the construction of `Chain` handles and the call sites of `postTx`
+* We need to extract the state handling (similar to the event queue) out of the `HydraNode` handle and shuffle the main of `hydra-node` a bit to be able to provide the latest `ChainState` to the chain callback as a continuation.
+* We need to make the `ChainState` already serializable (`ToJSON`, `FromJSON`) as it will be part of the `HeadState`.
+* We can drop the `TVar` of keeping `OnChainHeadState` in the `Hydra.Chain.Direct` module.
+* We might be able to simplify the `ChainState tx` to be just a `UTxOType tx` later.
+
+## Alternative
+
+* We could extend `PostChainTx` and `OnChainTx` with `ChainState` and keep the signatures:
+```hs
+postTx :: MonadThrow m => PostChainTx tx -> m ()
+type ChainCallback tx m = (ChainState tx -> Maybe (OnChainTx tx) -> m ()
+```
+* Consequences:
+  - We need to change the interface between `Hydra.Chain.Direct.Tx` and `Hydra.Chain.Direct.State` to something else than `OnChainTx` as the former would not have a suitable `ChainState` to construct, e.g. `observeInitTx` would return `Maybe InitObervation`. This may be an orthogonally desired change though.
+  - Traces (e.g. `ToPost`) and errors (e.g. `PostTxError`) would automatically include the full `ChainState`, which might be helpful but also possible big.