Skip to content
This repository has been archived by the owner on Jan 24, 2022. It is now read-only.

Adds a new section to the documentation on "proxy patterns" #288

Merged
merged 8 commits into from
Oct 22, 2018
40 changes: 0 additions & 40 deletions packages/docs/docs/docs/advanced.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,46 +59,6 @@ upgradeable! Even if you find a bug after deploying them to mainnet, you will
be able to fix it without losing the contract state and in a way that's
transparent for your users.

## The proxy system
The upgradeability system in ZeppelinOS is based on a proxy system: for each deployed contract implementation (the _logic contract_), another, user-facing contract is deployed as well (the _proxy_). The proxy will be the one in charge of the contract's storage, but will forward all function calls to the backing logic contract. The only exception to this are calls made by the owner of the proxy for administrative purposes, which will be handled by the proxy itself.

The way the proxy forwards calls to the logic contract relies on [`delegatecall`](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-7.md), the mechanism the EVM provides to execute foreign code on local storage. This is normally used for libraries such as [`SafeMath`](https://github.com/OpenZeppelin/openzeppelin-solidity/blob/master/contracts/math/SafeMath.sol), which provide useful functionality but have no storage. ZeppelinOS, however, exploits this mechanism to provide upgradeability: a user only interacts with the proxy, and, when a new logic contract is available, the proxy owner simply points it to the upgraded contract. All of this is achieved in a way that is transparent for the user, as the proxy address is always the same.

If you want to find out more about different possible proxy patterns, be sure to check [this post](https://blog.zeppelinos.org/proxy-patterns/).



## Preserving the storage structure
As mentioned in the [Building upgradeable applications](building.md) guide, when upgrading your contracts, you need to make sure that all variables declared in prior versions are kept in the code. New variables must be declared below the previously existing ones, as such:

```sol
contract MyContract_v1 {
uint256 public x;
}

contract MyContract_v2 {
uint256 public x;
uint256 public y;
}
```

Note that this must be so _even if you no longer use the variables_. There is no restriction (apart from gas limits) on including new variables in the upgraded versions of your contracts, or on removing or adding functions.

This restriction is due to how [Solidity uses the storage space](https://solidity.readthedocs.io/en/v0.4.21/miscellaneous.html#layout-of-state-variables-in-storage). In short, the variables are allocated storage space in the order they appear (for the whole variable or some pointer to the actual storage slot, in the case of dynamically sized variables). When we upgrade a contract, its storage contents are preserved. This entails that if we remove variables, the new ones will be assigned storage space that is already occupied by the old variables.

We are working to automatically detect any incompatible modifications to the storage structure, but for the time being, you need to perform this check manually whenever you introduce a new change.

## Initializers vs. constructors
As we saw in the [Building upgradeable applications](building.md) guide, we did not include a constructor in our contracts, but used instead an `initialize` function. The reason for this is that constructors do not work as regular functions: they are invoked once upon a contract's creation, but their code is never stored in the blockchain. This means that they cannot be called from the contract's proxy as we call other functions. Thus, if we want to initialize variables in the _proxy's storage_, we need to include a regular function for doing so.

The ZeppelinOS CLI provides a way for calling this function and passing it the necessary arguments when creating the proxy:

```
zos create MyContract --init <initializingFunction> --args <arguments> --network <network>
```

where `<initializingFunction>` is the name of the initializing function (marked with an `initializer` modifier in the code), and `<arguments>` is a comma-separated list of arguments to the function.

#### Calling Initialize Functions Manually in Your Unit Tests

Truffle does not know how to resolve situations where a contract has functions that have matching names, but different arities. Here's an example of a `TimedCrowdsale` contract that inherits from `Crowdsale` which results in a contract that has two `initialize` functions with different arities:
Expand Down
150 changes: 150 additions & 0 deletions packages/docs/docs/docs/proxies.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
---
id: proxies
title: Proxy Pattern
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to brand this:
The ZeppelinOS Upgrade Pattern

The problem here is that the ideas for this come from many different places, so maybe at the end we can add some generic thanks to everybody researching about upgradeability in ethereum.

@martriay what do you think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keep in mind that the title has to fit in the sidebar =D

---

This article describes the "unstructured storage" proxy pattern, the fundamental building block of ZeppelinOS's upgrades.

Note: For a more in depth read, please see [blog.zeppelinos.org/proxy-patterns](https://blog.zeppelinos.org/proxy-patterns/), which discusses the need for proxies, goes into more technical detail on the subject, elaborates on other possible proxy patterns that were considered for zOS, and more.

## Why upgrade a contract?

By design, smart contracts are immutable. On the other hand, software quality heavily depends on the ability to upgrade and patch source code in order to produce iterative releases. Even though blockchain based software profits significantly from the technology's immutability, still a certain degree of mutability is needed for bug fixing and potential product improvements. ZeppelinOS solves this apparent contradiction by providing an easy to use, simple, robust, and opt-in upgrade mechanism for smart contracts that can be controlled by any type of governance, be it a multi-sig wallet, a simple address or a complex DAO.

## Upgrading via the proxy pattern

The basic idea behind using a proxy for upgrades. The first contract is a simple wrapper or "proxy" which users interact with directly and is in charge of forwarding transactions to and from the second contract, which contains the logic. The key concept to understand is that the logic contract can be replaced while the proxy, or the access point is never changed. Both contracts are still immutable in the sense that their code cannot be changed, but the logic contract can simply be swapped by another contract. The wrapper can thus point to a different logic implementation and in doing so, the software is "upgraded".

User ---- tx ---> Proxy ----------> Implementation_v0
|
------------> Implementation_v1
|
------------> Implementation_v2

## Proxy forwarding

The most immediate problem that proxies need to solve is how the proxy exposes the entire interface of the logic contract without requiring a one to one mapping of the entire logic contract's interface. That would be difficult to maintain, prone to errors, and would make the interface itself not upgradeable. Hence, a dynamic forwarding mechanism is required. The basics of such mechanism are presented in the code below:

```solidity
assembly {
let ptr := mload(0x40)
calldatacopy(ptr, 0, calldatasize) // (1) copy incoming call data
let result := delegatecall(gas, _impl, ptr, calldatasize, 0, 0) // (2) forward call to logic contract
let size := returndatasize
returndatacopy(ptr, 0, size) // (3) retrieve return data

switch result
case 0 { revert(ptr, size) }
default { return(ptr, size) } // (4) forward return data back to caller
}
```

This code can be put in the [fallback function](https://solidity.readthedocs.io/en/v0.4.21/contracts.html#fallback-function) of a proxy, and will forward any call to any function with any set of parameters to the logic contract without it needing to know anything in particular of the logic contract's interface. In essence, (1) the `calldata` is copied to memory, (2) the call is forwarded to the logic contract, (3) the return data from the call to the logic contract is retrieved, and (4) the returned data is forwarded back to the caller. The technique needs to be implemented using Yul because [Solidity's `delegatecall`](https://solidity.readthedocs.io/en/v0.4.21/introduction-to-smart-contracts.html#delegatecall-callcode-and-libraries) returns a boolean instead of the callee's return data.

A very important thing to note is that the code makes use of the EVM's `delegatecall` opcode which executes the callee's code in the context of the caller's state. That is, the logic contract controls the proxy's state and the logic contract's state is meaningless. Thus, the proxy doesn't only forward transactions to and from the logic contract, but also represents the pair's state. The state is in the proxy and the logic is in the particular implementation that the proxy points to.

## Unstructured storage proxies

A problem that quickly comes up when using proxies has to do with the way in which variables are stored in the proxy contract. Suppose that the proxy stores the logic contract's address in it's only variable `address public _implementation;`. Now, suppose that the logic contract is a basic token whose first variable is `address public _owner`. Both variables are 32 byte in size, and as far as the EVM knows, occupy the first slot of the resulting execution flow of a proxied call. When the logic contract writes to `_owner`, it does so in the scope of the proxy's state, and thus really writes to `_implementation`. This problem can be referred to as a "storage collision".

|Proxy |Implementation |
|--------------------------|-------------------------|
|address _implementation |address _owner | <=== Storage collision!
|... |mapping _balances |
| |uint256 _supply |
| |... |

There are many ways to overcome this problem, and the "unstructured storage" approach which ZeppelinOS implements works as follows. Instead of storing the `_implementation` address at the proxy's first storage slot, it chooses a pseudo random slot instead. This slot is sufficiently random, that the probability of a logic contract declaring a variable at the same slot is negligible. The same principle of randomizing slot positions in the proxy's storage is used in any other variables the proxy may have, such as an admin address (that is allowed to update the value of `_implementation`), etc.

|Proxy |Implementation |
|--------------------------|-------------------------|
|... |address _owner |
|... |mapping _balances |
|... |uint256 _supply |
|... |... |
|... | |
|... | |
|... | |
|... | |
|address _implementation | | <=== Randomized slot.
|... | |
|... | |

An example of how the randomized storage is achieved:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't use the word random, it sounds not-robust, I'd rather use something like fixed, explaining that we choose a specific position making sure that Solidity won't ever use it


```
bytes32 private constant implementationPosition = keccak256("org.zeppelinos.proxy.implementation");
```

As a result, a logic contract doesn't need to care about overwriting any of the proxy's variables. Other proxy implementations that face this problem usually imply having the proxy know about the logic contract's storage structure and adapt to it, or instead having the logic contract know about the proxy's storage structure and adapt to it. This is why this approach is called "unstructured storage"; neither of the contracts needs to care about the structure of the other.

## Storage collisions between implementation versions

As discussed, the unstructured approach avoids storage collisions between the logic contract and the proxy. However, storage collisions between different versions of the logic contract can occur. In this case, imagine that the first implementation of the logic contract stores `address public _owner` at the first storage slot and an upgraded logic contract stores `address public _lastContributor` at the same first slot. When the updated logic contract attempts to write to the `_lastContributor` variable, it will be using the same storage position where the previous value for `_owner` was being stored, and overwrite it!

#### Incorrect storage preservation:

|Implementation_v0 |Implementation_v1 |
|--------------------|-------------------------|
|address _owner |address _lastContributor | <=== Storage collision!
|mapping _balances |address _owner |
|uint256 _supply |mapping _balances |
|... |uint256 _supply |
| |... |

#### Correct storage preservation:

|Implementation_v0 |Implementation_v1 |
|--------------------|-------------------------|
|address _owner |address _owner |
|mapping _balances |mapping _balances |
|uint256 _supply |uint256 _supply |
|... |address _lastContributor | <=== Storage extension.
| |... |

The unstructured storage proxy mechanism doesn't safeguard against this situation. It is up to the user to have new versions of a logic contract extend previous versions, or otherwise guarantee that the storage hierarchy is always appended to but not modified. ZeppelinOS' CLI does however| Tables | Are | Cool |
| ------------- |:-------------:| -----:|
| col 3 is | right-aligned | $1600 |
| col 2 is | centered | $12 | detect such collisions, and warns the developer appropriate.

## The constructor caveat

In Solidity, code that is inside a constructure or part of a global variable declaration is not part of a deployed contract's runtime bytecode. This code is executed only once, when the contract instance is deployed. As a consequence of this, the code within a logic contract's constructor will never be executed in the context of the proxy's state. To rephrase, proxies are completely oblivious to the existance of constructors. It's simply as if they weren't there for the proxy.

The problem is easily solved though. Logic contracts should move the code within the constructor to a regular 'initializer' function, and have this function be called whenever the proxy links to this logic contract. Special care needs to be taken with this initializer function so that it can only be called once, which is one of the properties of constructors in general programming.

This is why when the zOS CLI creates a proxy, it allows you to indicate an initializer function:

```bash
npx zos create MyLogicContract --init initialize --args arg1,arg2,arg3
```

With this command, ZeppelinOS creates a proxy that wraps around `MyLogicContract`, uses `MyLogicContract` as the logic contract, and calls the logic contract's `initialize` function.

To ensure that the `initialize` function can only be called once, a simple modifier is used. ZeppelinOS provides this functionality via a contract that can be extended:

```solidity

import "zos-lib/contracts/Initializable.sol";

contract MyContract is Initializable {

function initialize(address arg1, uint256 arg2, bytes arg3) initializer public payable {
// "constructor" code...
}

}

```

Notice how the contract extends `Initializable` and implements the `initializer` provided by it.

## Summary

Any developer using zOS should be familiar with proxies in the ways that are described in this article. In the end, the concept is very simple, and zOS is designed to encapsulate all the proxy mechanics in a way that the amount of things you need to keep in mind when developing upgradeable applications are reduced to an absolute minimum. It all comes down to a 3 item list:

* Have a basic understanding of what a proxy is
* Always extend storage instead of modifying it
* Make sure your contracts use initializer functions instead of constructors

Furthermore, ZeppelinOS will let you know when something goes wrong with one of the items in this list. So, seat back, enjoy, code, and let zOS take care of the rest.