-
Notifications
You must be signed in to change notification settings - Fork 204
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat(core): support multiple indy ledgers (#474)
Signed-off-by: Timo Glastra <[email protected]>
- Loading branch information
1 parent
ed9db11
commit 47149bc
Showing
31 changed files
with
1,221 additions
and
266 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,49 +1,43 @@ | ||
| # Ledger | ||
|
|
||
| > TODO | ||
| > | ||
| > - Context, some explanations | ||
| > - Why use public networks or local development networks | ||
| > - Whats the difference between the different public networks | ||
| - [Using Public Test Networks](#using-public-test-networks) | ||
| - [Using Your Own Development Network](#using-your-own-development-network) | ||
| - [VON Network](#von-network) | ||
| - [Preparing for Credential Issuance](#preparing-for-credential-issuance) | ||
| - [DID](#did) | ||
| - [Schema](#schema) | ||
| - [Credential Definition](#credential-definition) | ||
|
|
||
| ## Using Public Test Networks | ||
|
|
||
| > TODO | ||
| > | ||
| > - For development you can use one of the public test networks | ||
| > - Sovrin BuilderNet: https://selfserve.sovrin.org/ | ||
| > - BCGov Test Network: http://dev.greenlight.bcovrin.vonx.io/ | ||
| > - Indicio Test Network: https://selfserve.indiciotech.io/ | ||
| ## Using Your Own Development Network | ||
|
|
||
| ### VON Network | ||
|
|
||
| > TODO: | ||
| > | ||
| > - [VON Network](https://github.com/bcgov/von-network) install steps | ||
| > - con: only works if the framework runs in a docker container | ||
| ## Preparing for Credential Issuance | ||
|
|
||
| > TODO | ||
| ### DID | ||
|
|
||
| > TODO | ||
| ### Schema | ||
|
|
||
| > TODO | ||
| ### Credential Definition | ||
|
|
||
| > TODO | ||
| - [Configuration](#configuration) | ||
| - [Pool Selector Algorithm](#pool-selector-algorithm) | ||
|
|
||
| ## Configuration | ||
|
|
||
| Ledgers to be used by the agent can be specified in the agent configuration using the `indyLedgers` config. Only indy ledgers are supported at the moment. The `indyLedgers` property is an array of objects with the following properties. Either `genesisPath` or `genesisTransactions` must be set, but not both: | ||
|
|
||
| - `id`\*: The id (or name) of the ledger, also used as the pool name | ||
| - `isProduction`\*: Whether the ledger is a production ledger. This is used by the pool selector algorithm to know which ledger to use for certain interactions (i.e. prefer production ledgers over non-production ledgers) | ||
| - `genesisPath`: The path to the genesis file to use for connecting to an Indy ledger. | ||
| - `genesisTransactions`: String of genesis transactions to use for connecting to an Indy ledger. | ||
|
|
||
| ```ts | ||
| const agentConfig: InitConfig = { | ||
| indyLedgers: [ | ||
| { | ||
| id: 'sovrin-main', | ||
| isProduction: true, | ||
| genesisPath: './genesis/sovrin-main.txn', | ||
| }, | ||
| { | ||
| id: 'bcovrin-test', | ||
| isProduction: false, | ||
| genesisTransactions: 'XXXX', | ||
| }, | ||
| ], | ||
| } | ||
| ``` | ||
|
|
||
| ### Pool Selector Algorithm | ||
|
|
||
| The pool selector algorithm automatically determines which pool (network/ledger) to use for a certain operation. For **write operations**, the first pool is always used. For **read operations** the process is a bit more complicated and mostly based on [this](https://docs.google.com/document/d/109C_eMsuZnTnYe2OAd02jAts1vC4axwEKIq7_4dnNVA) google doc. | ||
|
|
||
| The order of the ledgers in the `indyLedgers` configuration object matters. The pool selection algorithm works as follows: | ||
|
|
||
| - When the DID is anchored on only one of the configured ledgers, use that ledger | ||
| - When the DID is anchored on multiple of the configured ledgers | ||
| - Use the first ledger (order of `indyLedgers`) with a self certified DID | ||
| - If none of the DIDs are self certified use the first production ledger (order of `indyLedgers` with `isProduction: true`) | ||
| - If none of the DIDs are self certified or come from production ledgers, use the first non production ledger (order of `indyLedgers` with `isProduction: false`) | ||
| - When the DID is not anchored on any of the configured ledgers, a `LedgerNotFoundError` will be thrown. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| import type { RecordTags, TagsBase } from '../storage/BaseRecord' | ||
|
|
||
| import { BaseRecord } from '../storage/BaseRecord' | ||
| import { uuid } from '../utils/uuid' | ||
|
|
||
| export type CustomCacheTags = TagsBase | ||
| export type DefaultCacheTags = TagsBase | ||
|
|
||
| export type CacheTags = RecordTags<CacheRecord> | ||
|
|
||
| export interface CacheStorageProps { | ||
| id?: string | ||
| createdAt?: Date | ||
| tags?: CustomCacheTags | ||
|
|
||
| entries: Array<{ key: string; value: unknown }> | ||
| } | ||
|
|
||
| export class CacheRecord extends BaseRecord<DefaultCacheTags, CustomCacheTags> { | ||
| public entries!: Array<{ key: string; value: unknown }> | ||
|
|
||
| public static readonly type = 'CacheRecord' | ||
| public readonly type = CacheRecord.type | ||
|
|
||
| public constructor(props: CacheStorageProps) { | ||
| super() | ||
|
|
||
| if (props) { | ||
| this.id = props.id ?? uuid() | ||
| this.createdAt = props.createdAt ?? new Date() | ||
| this.entries = props.entries | ||
| this._tags = props.tags ?? {} | ||
| } | ||
| } | ||
|
|
||
| public getTags() { | ||
| return { | ||
| ...this._tags, | ||
| } | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| import { inject, scoped, Lifecycle } from 'tsyringe' | ||
|
|
||
| import { InjectionSymbols } from '../constants' | ||
| import { Repository } from '../storage/Repository' | ||
| import { StorageService } from '../storage/StorageService' | ||
|
|
||
| import { CacheRecord } from './CacheRecord' | ||
|
|
||
| @scoped(Lifecycle.ContainerScoped) | ||
| export class CacheRepository extends Repository<CacheRecord> { | ||
| public constructor(@inject(InjectionSymbols.StorageService) storageService: StorageService<CacheRecord>) { | ||
| super(CacheRecord, storageService) | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,73 @@ | ||
| import type { CacheRepository } from './CacheRepository' | ||
|
|
||
| import { LRUMap } from 'lru_map' | ||
|
|
||
| import { CacheRecord } from './CacheRecord' | ||
|
|
||
| export class PersistedLruCache<CacheValue> { | ||
| private cacheId: string | ||
| private limit: number | ||
| private _cache?: LRUMap<string, CacheValue> | ||
| private cacheRepository: CacheRepository | ||
|
|
||
| public constructor(cacheId: string, limit: number, cacheRepository: CacheRepository) { | ||
| this.cacheId = cacheId | ||
| this.limit = limit | ||
| this.cacheRepository = cacheRepository | ||
| } | ||
|
|
||
| public async get(key: string) { | ||
| const cache = await this.getCache() | ||
|
|
||
| return cache.get(key) | ||
| } | ||
|
|
||
| public async set(key: string, value: CacheValue) { | ||
| const cache = await this.getCache() | ||
|
|
||
| cache.set(key, value) | ||
| await this.persistCache() | ||
| } | ||
|
|
||
| private async getCache() { | ||
| if (!this._cache) { | ||
| const cacheRecord = await this.fetchCacheRecord() | ||
| this._cache = this.lruFromRecord(cacheRecord) | ||
| } | ||
|
|
||
| return this._cache | ||
| } | ||
|
|
||
| private lruFromRecord(cacheRecord: CacheRecord) { | ||
| return new LRUMap<string, CacheValue>( | ||
| this.limit, | ||
| cacheRecord.entries.map((e) => [e.key, e.value as CacheValue]) | ||
| ) | ||
| } | ||
|
|
||
| private async fetchCacheRecord() { | ||
| let cacheRecord = await this.cacheRepository.findById(this.cacheId) | ||
|
|
||
| if (!cacheRecord) { | ||
| cacheRecord = new CacheRecord({ | ||
| id: this.cacheId, | ||
| entries: [], | ||
| }) | ||
|
|
||
| await this.cacheRepository.save(cacheRecord) | ||
| } | ||
|
|
||
| return cacheRecord | ||
| } | ||
|
|
||
| private async persistCache() { | ||
| const cache = await this.getCache() | ||
|
|
||
| await this.cacheRepository.update( | ||
| new CacheRecord({ | ||
| entries: cache.toJSON(), | ||
| id: this.cacheId, | ||
| }) | ||
| ) | ||
| } | ||
| } |
71 changes: 71 additions & 0 deletions
71
packages/core/src/cache/__tests__/PersistedLruCache.test.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| import { mockFunction } from '../../../tests/helpers' | ||
| import { CacheRecord } from '../CacheRecord' | ||
| import { CacheRepository } from '../CacheRepository' | ||
| import { PersistedLruCache } from '../PersistedLruCache' | ||
|
|
||
| jest.mock('../CacheRepository') | ||
| const CacheRepositoryMock = CacheRepository as jest.Mock<CacheRepository> | ||
|
|
||
| describe('PersistedLruCache', () => { | ||
| let cacheRepository: CacheRepository | ||
| let cache: PersistedLruCache<string> | ||
|
|
||
| beforeEach(() => { | ||
| cacheRepository = new CacheRepositoryMock() | ||
| mockFunction(cacheRepository.findById).mockResolvedValue(null) | ||
|
|
||
| cache = new PersistedLruCache('cacheId', 2, cacheRepository) | ||
| }) | ||
|
|
||
| it('should return the value from the persisted record', async () => { | ||
| const findMock = mockFunction(cacheRepository.findById).mockResolvedValue( | ||
| new CacheRecord({ | ||
| id: 'cacheId', | ||
| entries: [ | ||
| { | ||
| key: 'test', | ||
| value: 'somevalue', | ||
| }, | ||
| ], | ||
| }) | ||
| ) | ||
|
|
||
| expect(await cache.get('doesnotexist')).toBeUndefined() | ||
| expect(await cache.get('test')).toBe('somevalue') | ||
| expect(findMock).toHaveBeenCalledWith('cacheId') | ||
| }) | ||
|
|
||
| it('should set the value in the persisted record', async () => { | ||
| const updateMock = mockFunction(cacheRepository.update).mockResolvedValue() | ||
|
|
||
| await cache.set('test', 'somevalue') | ||
| const [[cacheRecord]] = updateMock.mock.calls | ||
|
|
||
| expect(cacheRecord.entries.length).toBe(1) | ||
| expect(cacheRecord.entries[0].key).toBe('test') | ||
| expect(cacheRecord.entries[0].value).toBe('somevalue') | ||
|
|
||
| expect(await cache.get('test')).toBe('somevalue') | ||
| }) | ||
|
|
||
| it('should remove least recently used entries if entries are added that exceed the limit', async () => { | ||
| // Set first value in cache, resolves fine | ||
| await cache.set('one', 'valueone') | ||
| expect(await cache.get('one')).toBe('valueone') | ||
|
|
||
| // Set two more entries in the cache. Third item | ||
| // exceeds limit, so first item gets removed | ||
| await cache.set('two', 'valuetwo') | ||
| await cache.set('three', 'valuethree') | ||
| expect(await cache.get('one')).toBeUndefined() | ||
| expect(await cache.get('two')).toBe('valuetwo') | ||
| expect(await cache.get('three')).toBe('valuethree') | ||
|
|
||
| // Get two from the cache, meaning three will be removed first now | ||
| // because it is not recently used | ||
| await cache.get('two') | ||
| await cache.set('four', 'valuefour') | ||
| expect(await cache.get('three')).toBeUndefined() | ||
| expect(await cache.get('two')).toBe('valuetwo') | ||
| }) | ||
| }) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| export * from './PersistedLruCache' | ||
| export * from './CacheRecord' | ||
| export * from './CacheRepository' |
Oops, something went wrong.