Skip to content

Commit

Permalink
Merge branch 'master' into 7192-oracle-error
Browse files Browse the repository at this point in the history
  • Loading branch information
mergify[bot] authored Mar 22, 2023
2 parents ca62d7a + 5933241 commit 3aa2b5d
Show file tree
Hide file tree
Showing 39 changed files with 1,008 additions and 284 deletions.
3 changes: 3 additions & 0 deletions codecov.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ comment:
show_carryforward_flags: false
coverage:
status:
patch:
default:
informational: true
project:
default:
informational: true
Expand Down
51 changes: 51 additions & 0 deletions packages/SwingSet/docs/vat-upgrade.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,13 +63,37 @@ During the upgrade phase, v2 code is obligated to re-define all durable Kinds cr

As a special case, the root object returned from v2's `buildRootObject()` is automatically associated with exportID `o+0` (see [How Liveslots Uses the Vatstore](../../swingset-liveslots/src/vatstore-usage.md#counters)) and is therefore also obligated to support the same methods as its predecessor. This means that the root object is effectively always durable, and should not be explicitly persisted.

### Zone API

The [zone API](https://github.com/Agoric/agoric-sdk/tree/master/packages/zone#readme) provides a unified model for creating the objects mentioned above, regardless of their backing storage:

* singleton objects created with `zone.exo()`
* instances created with a "make" function from `zone.exoClass()`
* multifaceted kit instances created with a "makeKit" function from `zone.exoClassKit()`

You can obtain individual zones implementing this API as follows:
* Heap objects in vat RAM
- `import { heapZone } from '@agoric/zone';`
* Virtual objects in disk-based storage
- `import { virtualZone } from '@agoric/zone/virtual.js';`
* Durable objects in disk-based storage
- zone API maker found at `import { makeDurableZone } from '@agoric/zone/durable.js';` and
- zone API backed by a durable map and created by `makeDurableZone(durableMap)`

## Durable State

The v2 code runs in a brand new JavaScript environment; nothing is carried over from the RAM image of the v1 vat. To fulfill its obligations, v1 must arrange to deliver data and imported object references to v2. This uses two mechanisms: durable storage, and the "baggage".

Vat code has access to three categories of collection objects, each of which offers both Map and Set collections in both strong and weak forms. The simplest category consists of "_heap_" collections provided by JavaScript as `Map`, `Set`, `WeakMap`, and `WeakSet`; their data is held only in RAM.
The second two categories are both referred to as "[Stores](../../swingset-liveslots/src/vatstore-usage.md#virtualdurable-collections-aka-stores)"; they are created by `makeScalarBigMapStore()`, `makeScalarBigWeakMapStore()`, `makeScalarBigSetStore()`, or `makeScalarBigWeakSetStore()`, and their contents are held in disk-based storage. What differentiates the second two categories from each other is use of the `durable` option: when it is false, the collection is "_[merely-]virtual_" and not preserved across upgrade, but when it is true, the collection is "_durable_" and **is** preserved. Durable collections can only hold durable objects.

The zone API exposes providers for these collections as `zone.mapStore(label)`,
`zone.setStore(label)`, `zone.weakMapStore(label)`, and
`zone.weakSetStore(label)`. They only create a new collection if the `label`
entry in the zone has not been used before. If you want to unconditionally
create a fresh, unnamed collection in the zone, you can use the providers
exposed under `zone.detached()`, such as `zone.detached().mapStore(label)`.

Heap and merely-virtual collections are _ephemeral_ and discarded during upgrade. More precisely, the v2 code has no way to reach anything but durable data, so even if the kernel did not delete the DB records, the v2 code could not ever read them.

The v2 code gets exactly one special object during the upgrade phase, currently known as "baggage". This is a durable Map (i.e., the kind of object returned from `makeScalarBigMapStore('label', { durable: true })`). All versions get access to the baggage: the v1 code should add data to it, so that the v2 code can read it back out. This provides the bridge between versions that allows v2 to assume responsibility for the obligations created by v1. It also provides a way for v1 to deliver authorities (in the form of imported object references) to v2, so v2 can talk to the world as if it were v1.
Expand Down Expand Up @@ -107,6 +131,16 @@ const FooI = M.interface('foo', fooMethodGuards);
const makeFoo = prepareExoClass(someDurableMap, 'foo', fooI, initFoo, fooMethods);
```

or with the zone API:

```js
import { M, makeDurableZone } from '@agoric/zone';
const FooI = M.interface('foo', fooMethodGuards);
// someDurableMap should generally be reachable from baggage.
const zone = makeDurableZone(someDurableMap);
const makeFoo = zone.exoClass('foo', fooI, initFoo, fooMethods);
```

The v1 code can also store imported objects (Presences) and plain data in a durable collection. Durable collections are themselves durable objects, so they can be nested:

```js
Expand All @@ -116,6 +150,13 @@ const childMap = makeScalarBigMapStore(childLabel, { durable: true });
parentMap.init('child', childMap);
```

or with the zone API:

```js
const parentMap = makeDurableZone(baggage).mapStore('parent');
const childMap = makeDurableZone(parentMap).mapStore('child');
```

The "baggage" is a special instance of `makeScalarBigMapStore`, with backing data is stored in a well-known per-vat location so each version can be given a reference. For every piece of data that v1 wrote into the baggage, v2 can read an equivalent item from the baggage it receives. However, any data associated with such items that v1 did _not_ write into baggage is lost -- v2 is a distinct process from v1, with its own independent heap and virtual memory.

While the baggage can use any suitable keys, at least one of the baggage keys should be a piece of plain data such as a string. The v2 vat starts out with nothing but baggage and a pile of source code, and source code can carry strings but not object references. So to allow v2 to get started, it must be able to do at least one `baggage.get('string')`. The value it retrieves from that initial call can be a durable object, which then might be usable as a second key. But without at least one plain-data key, v2 won't be able to extract anything from the baggage.
Expand All @@ -131,6 +172,16 @@ initializeFoo(fooData);
initializeBar(barData);
```

or with the zone API:

```js
const zone = makeDurableZone(baggage);
const fooData = zone.mapStore('foo data');
const barData = zone.mapStore('bar data');
initializeFoo(fooData);
initializeBar(barData);
```

## From Outside: the AdminNode Upgrade API

An upgrade is triggered by invoking the `upgrade` method of the vat's "adminNode" control facet. This facet is returned when the vat is first created (along with the vat's root object), so the original creator is often the initiator of upgrades later. But a common pattern is to hand that control facet to some sort of governance object. The ability to upgrade a vat is also the ability to control its behavior, and any users who expect some particular vat behavior (e.g. when auditing some contract code) must take into account the possibility of upgrade, which means they'll care about who exactly can cause an upgrade to occur.
Expand Down
4 changes: 3 additions & 1 deletion packages/SwingSet/src/controller/controller.js
Original file line number Diff line number Diff line change
Expand Up @@ -508,12 +508,14 @@ export async function makeSwingsetController(
* testTrackDecref?: unknown;
* warehousePolicy?: { maxVatsOnline?: number };
* }} runtimeOptions
* @param {Record<string, unknown>} deviceEndowments
* @typedef { import('@agoric/swing-store').KVStore } KVStore
*/
export async function buildVatController(
config,
argv = [],
runtimeOptions = {},
deviceEndowments = {},
) {
const {
kernelStorage = initSwingStore().kernelStorage,
Expand Down Expand Up @@ -551,7 +553,7 @@ export async function buildVatController(
}
const controller = await makeSwingsetController(
kernelStorage,
{},
deviceEndowments,
actualRuntimeOptions,
);
return harden({ bootstrapResult, ...controller });
Expand Down
17 changes: 12 additions & 5 deletions packages/SwingSet/src/devices/bridge/device-bridge.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,18 @@ function sanitize(data) {
}

/**
* @typedef {object} BridgeDevice
* @property {(dstID: string, obj: any) => any} callOutbound
* @property {(handler: { inbound: (srcID: string, obj: any) => void}) => void} registerInboundHandler
* @typedef {object} BridgeRoot
* An object representing a bridge device from which messages can be received
* via a handler and to which messages can be sent.
* For multiplexing, each inbound or outbound message is associated with a string identifying its channel.
* @property {(channelId: string, obj: any) => any} callOutbound
* @property {(handler: { inbound: (channelId: string, obj: any) => void }) => void} registerInboundHandler
* @property {() => void} unregisterInboundHandler
*/

/**
*
* @param {*} tools
* @returns {BridgeDevice}
* @returns {BridgeRoot}
*/
export function buildRootDeviceNode(tools) {
const { SO, getDeviceState, setDeviceState, endowments } = tools;
Expand All @@ -46,6 +49,10 @@ export function buildRootDeviceNode(tools) {
inboundHandler = handler;
setDeviceState(harden({ inboundHandler }));
},
unregisterInboundHandler() {
inboundHandler = undefined;
setDeviceState(harden({ inboundHandler }));
},
callOutbound(...args) {
// invoke our endowment of the same name, with a sync return value
const retval = callOutbound(...args);
Expand Down
2 changes: 1 addition & 1 deletion packages/SwingSet/src/vats/timer/vat-timer.js
Original file line number Diff line number Diff line change
Expand Up @@ -934,7 +934,7 @@ export const buildRootObject = (vatPowers, _vatParameters, baggage) => {
* replace the device), just in case that's useful someday
*
* @param {unknown} timerNode
* @returns {Promise<TimerService>}
* @returns {TimerService}
*/
const createTimerService = timerNode => {
timerDevice = timerNode;
Expand Down
54 changes: 45 additions & 9 deletions packages/SwingSet/test/bootstrap-relay.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ export const buildRootObject = () => {
const timer = buildManualTimer();
let vatAdmin;
const vatData = new Map();
const devicesByName = new Map();
const callLogsByRemotable = new Map();

const { encodePassable, decodePassable } = makePassableEncoding();

Expand All @@ -22,8 +24,14 @@ export const buildRootObject = () => {
vatData.set(name, { root });
}
}
for (const [name, device] of Object.entries(devices)) {
devicesByName.set(name, device);
}
},

getDevice: async deviceName =>
encodePassable(devicesByName.get(deviceName)),

getVatAdmin: async () => encodePassable(vatAdmin),

getTimer: async () => encodePassable(timer),
Expand All @@ -46,18 +54,46 @@ export const buildRootObject = () => {
},

/**
* Turns an object into a remotable by ensuring that each property is a function
* Derives a remotable from an object by mapping each object property into a method that returns the value.
* This function is intended to be called from testing code outside of the vat environment,
* but methods of the returned remotable are intended to be called from *inside* that environment
* and therefore do not perform argument/response translation.
*
* @param {string} label
* @param {Record<string, any>} methodReturnValues
* @param {Record<string, any>} encodedMethodReturnValues
*/
makeRemotable: (label, encodedMethodReturnValues) => {
const callLogs = [];
const makeGetterFunction = (value, name) => {
const getValue = (...args) => {
callLogs.push([name, ...args]);
return value;
};
return getValue;
};
const methodReturnValues = decodePassable(encodedMethodReturnValues);
// `objectMap` hardens its result, but...
const methods = objectMap(methodReturnValues, makeGetterFunction);
// ... `Far` requires its methods argument not to be hardened.
const unhardenedMethods = { ...methods };
const remotable = Far(label, unhardenedMethods);
callLogsByRemotable.set(remotable, callLogs);
return encodePassable(remotable);
},

/**
* Returns a copy of a remotable's logs.
*
* @param {object} encodedRemotable
*/
makeSimpleRemotable: (label, methodReturnValues) =>
// braces to unharden so it can be hardened
encodePassable(
Far(label, {
...objectMap(methodReturnValues, v => () => decodePassable(v)),
}),
),
getLogForRemotable: encodedRemotable => {
const remotable = decodePassable(encodedRemotable);
const decodedLogs =
callLogsByRemotable.get(remotable) ||
Fail`logs not found for ${q(remotable)}`;
// Return a copy so that the original remains mutable.
return encodePassable(harden([...decodedLogs]));
},

messageVat: async ({ name, methodName, args = [] }) => {
const vat = vatData.get(name) || Fail`unknown vat name: ${q(name)}`;
Expand Down
2 changes: 1 addition & 1 deletion packages/SwingSet/tools/passableEncoding.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ const noop = () => {};

// TODO define these somewhere more accessible. https://github.com/endojs/endo/issues/1488
/**
* @typedef {Promise | import('@agoric/internal').Remotable} PassByRef
* @typedef {import('@endo/eventual-send').FarRef<unknown>} PassByRef
* Gets transformed by a marshaller encoding.
* As opposed to pass-by-copy
*/
Expand Down
3 changes: 2 additions & 1 deletion packages/internal/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,8 @@
"lint:types": "tsc -p jsconfig.json"
},
"dependencies": {
"@endo/eventual-send": "^0.16.8",
"@agoric/zone": "^0.1.0",
"@endo/far": "^0.2.14",
"@endo/marshal": "^0.8.1",
"@endo/promise-kit": "^0.2.52",
"jessie.js": "^0.3.2"
Expand Down
122 changes: 122 additions & 0 deletions packages/internal/src/callback.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
// @ts-check
import { E } from '@endo/far';

/**
* @template {(...args: unknown[]) => any} I
* @typedef {import('./types').Callback<I>} Callback
*/

/**
* @template {(...args: unknown[]) => any} I
* @typedef {import('./types').SyncCallback<I>} SyncCallback
*/

/**
* Synchronously call a callback.
*
* @template {(...args: unknown[]) => any} I
* @param {SyncCallback<I>} callback
* @param {Parameters<I>} args
* @returns {ReturnType<I>}
*/
export const callSync = (callback, ...args) => {
const { target, method, bound } = callback;
if (method === undefined) {
return target(...bound, ...args);
}
return target[method](...bound, ...args);
};

/**
* Eventual send to a callback.
*
* @template {(...args: unknown[]) => any} I
* @param {Callback<I>} callback
* @param {Parameters<I>} args
* @returns {Promise<Awaited<ReturnType<I>>>}
*/
export const callE = (callback, ...args) => {
const { target, method, bound } = callback;
if (method === undefined) {
return E(target)(...bound, ...args);
}
return E(target)[method](...bound, ...args);
};

/**
* Create a callback from a near function.
*
* @template {(...args: unknown[]) => any} I
* @template {(...args: [...B, ...Parameters<I>]) => ReturnType<I>} [T=I]
* @template {unknown[]} [B=[]]
* @param {T} target
* @param {B} bound
* @returns {SyncCallback<I>}
*/
export const makeSyncFunctionCallback = (target, ...bound) => {
/** @type {unknown} */
const cb = harden({ target, bound });
return /** @type {SyncCallback<I>} */ (cb);
};
harden(makeSyncFunctionCallback);

/**
* Create a callback from a potentially far function.
*
* @template {(...args: unknown[]) => any} I
* @template {import('@endo/far').ERef<
* (...args: [...B, ...Parameters<I>]) => ReturnType<I>
* >} [T=import('@endo/far').ERef<I>]
* @template {unknown[]} [B=[]]
* @param {T} target
* @param {B} bound
* @returns {Callback<I>}
*/
export const makeFunctionCallback = (target, ...bound) => {
/** @type {unknown} */
const cb = harden({ target, bound });
return /** @type {Callback<I>} */ (cb);
};
harden(makeFunctionCallback);

/**
* Create a callback from a near method.
*
* @template {(...args: unknown[]) => any} I
* @template {PropertyKey} P
* @template {{
* [x in P]: (...args: [...B, ...Parameters<I>]) => ReturnType<I>
* }} [T={ [x in P]: I }]
* @template {unknown[]} [B=[]]
* @param {T} target
* @param {P} methodName
* @param {B} bound
* @returns {SyncCallback<I>}
*/
export const makeSyncMethodCallback = (target, methodName, ...bound) => {
/** @type {unknown} */
const cb = harden({ target, method: methodName, bound });
return /** @type {SyncCallback<I>} */ (cb);
};
harden(makeSyncMethodCallback);

/**
* Create a callback from a potentially far method.
*
* @template {(...args: unknown[]) => any} I
* @template {PropertyKey} P
* @template {import('@endo/far').ERef<{
* [x in P]: (...args: [...B, ...Parameters<I>]) => ReturnType<I>
* }>} [T=import('@endo/far').ERef<{ [x in P]: I }>]
* @template {unknown[]} [B=[]]
* @param {T} target
* @param {P} methodName
* @param {B} bound
* @returns {Callback<I>}
*/
export const makeMethodCallback = (target, methodName, ...bound) => {
/** @type {unknown} */
const cb = harden({ target, method: methodName, bound });
return /** @type {Callback<I>} */ (cb);
};
harden(makeMethodCallback);
Loading

0 comments on commit 3aa2b5d

Please sign in to comment.